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ABSTRACT 


The  particular  problem  with  which  tire  research  was  concerned 
was  the  development  of  a  technique  to  discriminate  between  coughs 
and  other  audible  phenomena  which  originate  in  a  hospital  environment. 
Pattern  recognition  provided  such  a  technique.  Experimental  data  was 
available  in  the  form  of  audio  tape  recordings. 

Implementation  of  a  Bayes  categorization  decision  requires 
knowledge  of  the  underlying  conditional  joint  probability  density 
functions  of  the  measures  which  typify  the  patterns  to  he  recognized. 

An  adaptive  pattern  classifier  model  was  presented  which  circumvented 
the  difficulty  of  estimating  these  functions.  The  model  is  generally 
applicable  to  the  two-class  case  in  which  the  patterns  to  be  classified 
consist  of  sequential  segments  of  data  known  to  have  originated  from 
the  same  class.  The  model  took  the  form  of  a  layered  machine.  The 
first  stage  was  a  minimum  distance  classifier  with  respect  to  point  sets 
while  the  second  stage  utilized  the  first  stage  binary  valued  outputs  to 
implement  a  Bayes  decision. 

The  feature  extraction  and  measure  selection,  problems  were 
examined  experimentally,  feature  calculation  algorithms  wore  devel¬ 
oped  which  are  generally  applicable-  to  time  varying  signals.  The 
effectiveness  of  an  algorithm  for  selection  of  a  set  of  candidate  measures 


was  verified. 


Experimental  results  indicated  that,  for  the  particular  data  with 
which  the  research  was  concerned,  an  assumption  of  Markoff -l  dependency 
between  sequential  first  stage  decisions  of  the  pattern  classifier  was 
warranted.  A  pattern  classifier  which  was  based  on  this  assumption 
classified  97.9%  of  the  patterns  presented  to  its  input  correctly. 
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CHAPTER  I 


INTRODUCTION 

A  group  at  The  University  of  Texas  Southwestern  Medical  School  in 

Dallas,  Texas  is  engaged  in  research  in  the  general  area  of  respiratory 

* 

diseases.  One  parameter  of  importance  in  their  studies  is  the  number  of 
times  that  a  patient  coughs  during  a  given  time  interval.  The  primary  ob¬ 
jective  of  the  research  outlined  in  this  report  is  to  develop  ■  ’'.nique 
which  may  be  mechanized,  either  as  special  purpose  circuitry  to  operate 
in  a  real-time  environment  or  as  a  general  purpose  computer  program,  to 
obtain  this  parameter. 

The  results  may  find  additional  application  in  pharmaceutical 
evaluation.  Experience  indicates  that  self-evaluation  of  cough  activity 
by  the  patient  is  subjective  to  a  pronounced  degree.  In  the  evaluation 
of  a  drug  proposed  as  a  cough  alleviating  agent,  an  objective  method  of 
evaluation  of  the  results  of  administration  of  the  drug  is  necessary. 

for  the  purposes  oi  this  investigation  audible  phenomena  which 
occur  in  the  hospital  environment,  but  which  are  not  coughs,  will  be 
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referred  to  as  artifacts.  A  preliminaiy  attempt  by  personnel  at  Southwestern 
Medical  School  to  identify  coughs  by  means  of  an  empirically  adjusted 
filter  and  voice  controlled  relay  were  unsuccessful  .  When  adjusted  for 
actuation  by  coughs,  the  system  also  responds  to  many  artifacts.  This 
indicated  that  more  powerful  classifying  techniques  would  be  required. 

The  purpose  of  the  outlined  lesearch  was  to  develop  a  means  for 
discriminating  between  coughs  and  artifacts.  Pattern  lecognition  pro¬ 
vides  such  a  technique. 

During  recent  years  a  good  deal  of  effort  has  been  directed  toward 
the  study  oi  adpative  pattern  recognizers.  These  studies  fall  into  two 
broad  categories.  The  bulk,  of  the  literature  in  the  field  consists  of 
mathematical  studies  indicating  that  a  given  procedure  is  optimal,  or 
converges  to  an  optimal  procedure  for  a  specified  constraint  under  a 
given  set  oi  presumptions  regarding  underlying  probability  density 
functions.  The  second  approach  that  is  taken  is  to  apply  pattern  re¬ 
cognition  techniques  to  a  set  of  data  originating  from  a  particular 
experiment  for  which  the  underlying  probability  functions  are  unknown 
or  cannot  be  approximated  by  a  computationally  managable  analytical 
expression  and  for  which  optimality  cannot  therefore  be  shown.  The 
research  outlined  in  this  investigation  falls  into  the  latter  category. 

The  basics  of  pattern  recognition  are  outlined  in  Chapter  2  of  the 
report  with  emphasis  placed  on  those  portions  of  the  theory  that 
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bear  on  the  particular  pattern  lecognition  machine  selected  for  the  described 
research.  Data  acquisition  techniques  and  processing  used  to  acquire  the 
measures  utilized  in  the  recognition  process  and  the  basis  on  which  the 
particular  meaurcs  chosen  were  selected  ore  outlined  in  Chapter  3.  The 
mach  no  chosen  to  implement  the  decision  process  is  described  in  Chapter  4. 


Experimental  results  are  presented  in  Chapter  5. 


CHAPTER  II 


GENERAL  PATTERN  RECOGNITION  MODEL 

2 . 1  Basic  Model 

A  general  review  of  pertinent  portions  of  pattern  recognition  theory 
are  presented  m  this  chapter  ol  lire  report.  The  notation  employed  is( 
for  the  most  part,  that  used  by  N  ill  son. 

A  pattern  classifier  has  as  an  input  a  set  ol  ordered  (for  our  pur¬ 
poses,  real)  numbers  gleaned  Iroin  an  experiment.  These  numbers  will  be 
referred  to  as  measures  and  quantitatively  describe  an  event  It  will  be 
convenient  to  visualize  a  set  of  d  measures  6s  a  point  in  d— dimensional 
Euclidian  space  L^.  This  space  will  be  referred  to  as  "pattern  space". 
The  rectangular  coordinates  in  the  point  are  the  real  numbers  x  i,  xz  ,  ... 
x  .  The  vector  X  extending  from  the  origin  to  the  point  (x  i  ,  xs  ,  .  .  .  x^) 
v.d’.l  also  be  used  to  represent  the  pattern.  X  will  be  usee  to  designate 
uoth  the  point  and  the  vecloi  .  l’oi  computational  purposes  it  will  be 
considered  a  column  matrix. 

The  outpu'  of  the  pattern  classifier  is  a  decision  as  to  which  of 
R  classes  the  event  described  by  the  measures  belongs.  R  >  2.  The 
pattern  recognizer,  then,  is  a  device  which  maps  lire  points  of  L^  into 
the  category  numbers  1,  2,  .  .  .  ,  R.  Presuming  points  in  L^  which  map 
into  different  categories  occupy  disjoint  regions  ui  L^,  one  may  visualize 
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a  partitioning  of  the  space  by  surfaces  arranged  so  that  they  separate  the 
point  sets  belonging  to  the  different  categories.  These  ate  called  decision 
surfaces.  If  the  point  sets  are  net  disjoint,  decision  surfaces  may  still  be 
placed  in  accordance  with  some  predetermined  criteria. 

A  pattern  classifier  is  "adaptive”  if  it  has  tire  capability  of  modify¬ 
ing  its  performance  (repositioning  the  decision  surfaces)  in  accordance  with 
measures  presented  to  its  input.  The  process  of  modification  of  the  perfor¬ 
mance  of  the  machine  is  termed  "learning"  while  the  act  of  aiding  this 
learning  is  called  "training".  A  set  of  data,  chosen  as  typical ,  used  to 
accomplish  the  training  is  called  a  "tiainmg  set". 

The  mariner  in  which  training  is  conducted  gives  rise  to  two  broad 
classifications  of  learning  called  " learning  with  a  teacher"  and  "learning 
without  a  teacher".  Learning  without  a  teacher  implies  that  the  machine 
modifies  its  performance  without  explicit  instruction  as  tu  the  class  member¬ 
ship  of  the  naming  date.  In  learning  with  a  teachoi  the  true  membership 
of  tliu  input  measures  is  known  a  priori. 

l'oi  the  former  type  of  machine,  training  may  take  place  throughout 
the  working  cycle  of  the  machine.  In  the  lattei  case,  tiaining  is  done 
pilot  to  the  use  of  the  machine  as  a  classifier.  The  type  of  machine  with 
which  this  npoit  is  concur  ned  utrlixos  learning  with  a  lea  chut . 

2.2  Discriminant  I’unclions 

Thu  decision  suiiacus  previously  described  may  be  implicitly  defined 
by  a  sot  of  functions  containing  l<  members  .  Define  g  i(X) ,  gf.  (X) ,  .  .  .  ,  y  (X) 
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surface  separating  contiguous  regions  i  and  j  is  defined  by: 

y .()<)- g.(X)  =  0  (2-2) 

Discriminant  functions  provide  a  means  for  convenient  imple¬ 
mentation  of  pattern  classification.  Tor  a  pattern  X  which  is  to  be 
classified  as  belonging  to  one  of  R  classes,  the  R  discriminant  functions 
are  calculated  and  the  pattern  is  assigned  to  the  category  for  which  the 
associated  discriminant  function  has  the  largest  value. 

A  special  case  of  particular  interest  is  that  where  R  =  2.  In  this 
case  the  partition  of  pattern  space  into  the  two  regions  is  called  a  dichotomy. 
The  associated  discriminant  functions  may  be  combined  to  yield: 


g  (x)  =  g  i  ix)  -  y  b  (x) 

g(X)  >0  =  X  e  class  1  (2-31 

g(X)  <0  =»  X  e  class  2 

g(x)  =  0  =»  undefined  or  arbitrary  classification 
Discriminant  functions  may  be  selected  in  a  variety  ol  ways  de¬ 
pending  primarily  upon  the  extent  of  a  priori  knowledge  of  the  patterns  to 
be  classified.  As  previously  indicated,  the  method  employed  in  the  described 
research  utilizes  training. 
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2.3  Training 

Training  methods  have  been  broadly  grouped  into  two  categories -- 
parametric  and  non -parametric . 

The  criteria  for  differentiation  between  these  two  types  of  training 

are  not  consistent  among  all  researchers  in  the  field.  Nilsson^,  for 

example,  classifies  as  parametric  any  training  which  assumes  that  the 

patterns  to  be  categorized  are  known  a  priori  to  be  characterized  by  a 

12 

set  of  parameters,  some  of  which  may  be  unknown.  Ssbestyen  ,  on  the 
other  hand,  reserves  the  parametric  label  for  patterns  originating  from  a 
random  process  with  an  underlying  probability  distribution  function  of 
known  form  which  is  described  by  a  set  of  parameters,  some  of  which 
may  be  unknown  and  which  arc  themselves  random  variables.  This  latter 
definition  is  an  extension  of  that  encountered  in  estimation  theory.^ 

In  either  case,  training  consists  of  computing  estimates  of  the 
unknown  parameters. 

An  important  example  of  non-parametric  pattern  classification  is 
that  where  the  pattern  space  is  deterministic  in  nature. 

1’or  the  purpose  of  this  report,  training  ptocedures  will  be 
labeled  as  probabilistic  oi  non -probabilistic  depending  upon  the  assump¬ 
tions  made  as  to  the  nature  of  the  pattern  space.  In  some  cases  the  line 


will  not  be  cleaily  drawn. 
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2.4  Machine  Structure 

2,4.1  Statistical  Decision  Theory 

If  the  pattern  space  is  probabilistic  and  the  underlying  probability 
density  functions  and  a  priori  probability  of  occurrences  of  the  various 
classes  are  known,  a  decision  criterion  may  be  derived  which  is  optimal 
under  a  given  constraint. 

Define: 

c(i/j)  =  cost  of  deciding  X  e  i  when  in 
actuality  X  e  j . 

p(j/X)  =  conditional  probability  that  X  c  j 
given  that  X  has  occurred. 

R  (2‘4) 

L  (i)  =  X  c(:/j)p(j/X)  =  conditional  average  loss 

-i  j=i 

for  the  decision  X  e  i; 
defined  a  priori. 

It  is  to  be  noted  that  X  is  a  vector.  p(j/X)  will  therefore  take  the 
form  of  a  conditional  joint  probability: 

P(j/X)  =  P(j/'xi  ,  xa . x  )  (2-5) 

A  deersion  which  minimizes  the  above  conditional  average  loss  is 
called  optimum  and  a  machine  which  implements  such  a  decision  is  called 
a  Bayes  machine. 

Examining  equation  (2-4),  it  is  seen  that  in  order  to  derive  an 
optimum  decision  a  set  of  costs  must  be  assigned  which  weight  the  re¬ 
lative  importance  of  ihe  different  errors  in  classification.  In  addition,  the 
a  posteriori  probabilities  p(j/X)  ,  j  =  1,  2,  .  .  .  ,  R  must  be  known.  This 
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latter  requirement  is  almost  never  accomplished  in  practice  and  would  be 
difficult  to  estimate  directly  from  experimental  observations.  Hayes' 
theorem,  however,  allows  the  calculation  of  the  a  posteriori  probabilities 
in  terms  of  estimable  quantities: 


p(X/j)  is  called  the 
into  (2-4): 


p(j/X)  =- 


p(x) 


"likelihood"  of  j  with  respect  to  X. 


(2-6) 

Substituting  (2-6) 


Lx(i)  = 


(2-7) 


It  is  seen  that  l/p(X)  is  a  common  term  for  all  j.  Minimization  of 


(2-7)  is  equivalent  to  minimization  of  (2-8): 


U)=  2  c(i/j)p(ii/j)p(j)  (2-8) 

A  j=l 


Equation  (2-8)  takes  a  form  particularly  amenable  to  calculation  if 
the  costs  of  incorrect  decisions  are  made  equal  while  the  cost  of  a  correct 
decision  is  set  at  zero: 


where  6  is  the  Kronecker  de’ta  function. 

ij 

Substitution  of  (2-9)  into  (2-8)  yields: 


l  (i)  =  X  p(X/j)p(j)  -  p(X/i)pO) 

X  j=]  (2-10) 

=  p(X)  -  pCv'i)p(i) 
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Minimization  of  (2-10)  over  al'l  i,  i  =  1,  2,  ...  ,  R  implies  maximiza¬ 
tion  of  p(X,/i)p(i),  1  —  l ,  2 ,  ...  ,  R.  The  discriminant  functions  for  the  R 
categories  would  therefore  be  the  quantities  p(X/i)p(i) ,  i  =  1,  2,  ...  ,  R, 
and  the  decision  made  would  be  in  accordance  with  discriminant  criteria 
outlined  in  section  2.2.  For  this  special  loss  function,  called  a  symmetrical 
loss  function,  the  decision  criterion  has  been  given  the  name  "Ideal  Observer" 
criterion  in  decision  theory.  It  can  be  shown  that  this  decision  rule 
minimizes  the  probability  of  erroneous  classification. 

In  the  event  that  the  a  priori  probabilities  p(i) ,  i  =  1 ,  2 ,  .  .  .  ,  R  are 
unknown  and  are  therefore  taken  as  being  equally  probable,  the  discriminant 
function  is  of  the  form  g^(X)  =  p(X/i)  and  the  criterion  is  known  as  the 
"Maximum  Likelihood"  criterion  in  decision  theory.  This  criterion  .implies 
that  X  originates  from  the  category  for  which  its  occurrence  is  most  prob¬ 
able. 

For  the  probabilistic  decision  criteria  outlined,  training  consists 
of  estimating  ths  various  probabilities.  Since  X  is  a  vector,  the  prob¬ 
abilities  p(X/i)  aie  joint  conditional  probabilities  and  must  be  estimated 
for  all  points  in  pattern  space.  This  is  a  formidable  task  if  the  dimension¬ 
ality  of  X  is  large  unless  some  simplifying  assumptions  are  made.  A  common 
assumption  that  is  made  is  that  the  underlying  probability  density  functions 
aie  Gaussian,  in  which  case  training  consists  of  estimating  the  unknown 
mean  and  covaiiance  matrices  of  the  density  functions.  This  line  of  attack 


will  not  be  pursued  since  the  measures  employed  do  not  exhibit  Gaussian 
properties  and  any  analytical  calculations  employing  this  assumption  would 
be  simply  a  mathematical  exercise. 

A  brief  summary  of  the  probabilistic  pattern  recognition  model  for 
the  symmetrical  loss  function  is  in  order  at  this  point.  The  input  to  the 
pattern  classifier  is  a  set  of  measures  which  is  represented  as  a  vector 
X  in  .  Training  consists  of  estimating  the  associated  likelihoods  and 
a  priori  probabilities  for  all  points  in  E  for  categories  i  =  1,  2,  ...  , 

R.  The  discriminant  function  for  category  i  is  g.(X)  =  p(X/i)p(i).  Classi¬ 
fication  of  a  vector  X  of  unknown  class  membership  is  accomplished  by 
evaluating  g^(X)  for  i  =  1,  2,  ...  ,  R  and  assigning  X  to  the  category  for 
which  the  associated  g(X)  is  largest.  Decision  surfaces  are  defined  by 
the  relationshps  g.(X);  i,  j  =  1,  2,  ...  ,R  --i/j  . 

2.4.2  Linear  Discriminants 

A  class  of  discriminant  functions  with  a  particularly  simple  physical 
implementation  is  the  linear  discriminant  function: 


d 

g .  (X)  =  E  w. .  x.  -t  w.  i  =  1 »  ,  ...  ,  R 

i  j  =  1  ij  ]  i,dtl 


(2-11) 


W  -  X  + 


i ,  d+ 1 


For  the  special  case  of  R  =  2  ,  the  relationship  of  equation  (2-3) 


applies.  Substituting  (2-11)  into  (2-3)  yields: 
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It  is  noted  that  the  relationship  W  •  X  =  0  defines  a  hyperplane  in  E 

which  passes  through  the  origin,  W-  X  +  w^  ^  —  0  defines  a  hyperpiano  with 

the  distance  to  the  origin  dependent  upon  |w|  and  w  ^  . 

It  wil1  be  convenient  at  this  point  to  define  an  augmented  vector 

X'  =  (x, ,  x„  ,  Equation  (2  -1  2)  then  takes  the  form: 

~  1  2  d 

g(X)  =  W-  X'  (2-13) 

It  is  to  be  noted  that,  although  X  and  W  are  vectors  in  E^4  ^ , 
equation  (2-13)  set  equal  to  zero  defines  a  dichotomizing  hyperplane  in 
E^.  A  pattern  set  for  which  a  hyperplane  exists  such  that  all  points 
belonging  to  category  1  are  on  one  side  of  the  hyperplane  and  all  points 
belonging  to  category  2  are  on  the  opposite  side  of  the  hyperplane  is  said 
to  be  linearly  separable.  Given  such  a  training  set,  training  consists  of 
finding  a  W  which  satisfies  the  following  relationship: 


g(X)  =  W-  X1  >  0  for  all  X  e  class  1 
<  0  for  all  X  e  class  2 


(2-14) 


A  device  capable  of  physically  implementing  equation  (2-14)  is 
the  threshold  logic  unit  (TLU) .  The  TLU  has  d  inputs  which  are  weighted 
by  multiplying  each  input  ny  a  constant.  The  weighted  inputs  are  then 
summed  and  fed  into  a  threshold  device,  li  the  summed  input  is  greater 
than  a  pre-set  bias  level,  the  output  of  the  threshold  device  is  a  high 
level;  if  the  summed  weighted  inputs  are  below  the  bias  level,  the  output 
of  the  threshold  device  is  a  low  level.  Tor  our  purposes,  the  high  level 
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output  corresponds  to  a  category  1  membership  decision  while  tine  low  level 

corresponds  to  a  category  2  membership  decision.  The  inputs  are  the  d 

measures  of  the  unaugmented  pattern  and  the  weighls  correspond  to  the  first 

d  components  of  W.  The  bias  level  is  associated  with  w 

—  d+ 1 

An  example  of  a  circuit  which  comprises  a  TLU  is  a  resistive  summing 

network  feeding  a  high  gain  amplifier  driven  into  saturation. 

Summarizing  the  properties  of  a  TLU,  the  dichotomizing  surface  in 

E  is  a  hvperplane  which  has  an  orientation  given  by  tire  weights  Wi  ,  ws  , 

...  ,  w  with  a  position  proportional  to  w,  ,  .  The  distance  from  the 
d  d+1 

hyperplane  to  an  arbitrary  pattern  X  (unaugmented)  is  proportional  to  the 
value  of  g(X) . 

2.4.3  Piecewise  Linear  Machines 


Another  concept  which  will  be  of  importance  in  the  outlined  research 
is  that  of  a  piecewise  linear  machine.  Minimum  distance  classifiers  with 
respect  to  point  sets  constitute  a  sub-class  of  piecewise  linear  machines 
and  will  be  used  for  illustrative  purposes. 

Suppose  that  it  has  been  determined  that  the  patterns  to  be  classifed 
cluster  (are  "close  to")  about  some  predetermined  sat  of  prototype  points. 

In  the  discussion  which  follows  X  will  be  unagumented.  Consider  the 
special  case  of  R  categories,  each  associated  with  a  single  point  p.,  i  =  l. 


m  *ri  rulr 


X  is  to  assign  X  to  that  class  i  for  which  the  distance  between  X  and  the 


14 


P.  is  less  than,  the  distance  between  X  and  any  other  P. .  Equivalently,  one 
— io  “  ~i 

may  compare  the  squared  distances. 

The  squared  Euclidian  distance  between  points  X  and  JP  is  given  by: 


(X  -  P.)2  =  (X  -  P.)-(X  -  P.) 
1  11 


(2-15) 


=  X-X  -  2X-P+  P.-  P, 


(2 -1C) 


It  is  seen  that  a  choice  of  the  minimum  squared  distance  is  equivalent 


to  ;:  e  selection  of  the  maximum: 

g  (X)  =  X ■  P  -  1/2  P-  P.  (2-17) 

l  - i  - i  — i 

Comparing  equation  (2-17)  with  equation  (2-11),  it  is  seen  that: 


w. , 
U 


Pij 


i  =  l,  2 ,  ...  ,  P 
i  =  1,  2 ,  ...  ,  d 


w  =  -1/2  P.-  P. 

i ,  d+  1  -i  ~r 

and  the  decision  criterion  is  as  previously  outlined. 


(2-18) 


Suppose  that,  rather  than  a  single  point  typifying  a  class,  L. 
prototype  points  are  associated  with  class  i,  i  =  l,  2,  ...  ,  R.  Wa  may 
define  the  R  discriminant  functions: 

g,(X)  =  max  [g.^(X)J 


=  max  [P.(j)-  X  -  1/2  P.(j)  •  P.(j)  ]  (2  -IS) 

j  =  1 .  2 . L 

i  =  1,  2 ,  ...  ,  R 

The  g^  (X)is  called  a  subsidiary  discriminant  function  and  is  seen  to  be  of  the  form  of 
equation  (2-17)  which  is  linear  in  X.  Since  each  of  the  g.(X)  is  a  piecewise 
linear  function  of  X  they  are  called  piecewise  linear  discriminant  functions. 
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The  decision  surfaces  w.iich  they  describe  are  sections  of  hyperplanos .  It 
is  noted  that  for  the  particular  constraints  imposed  that  the  decision  regions 
described  are  convex. 

2.4.4  Layered  Machines 

It  has  been  shown^thc.t  for  the  case  of  R  =  2,  a  special  class  of 
circuits  called  layered  machines  implement,  a  piecewise  linear  machine. 

A  layered  machine  is  a  network  of  TLUs  organized  in  layers  so  that 
the  inputs  to  each  bank  of  TLUs  are  the  outputs  oi  the  proceeding  bank,  with 
the  exception  of  the  first  bank,  whose  inputs  arc  the  pattern  to  be  classified. 
The  last  bank  consists  of  a  single  TLU . 

2.5  General  Model  Summary 

This  chapter  of  this  paper  has  reviewed  those  topics  in  pattern 
recognition  theory  which  have  a  direct  relation  to  the  particular  pattern 
classifier  which  was  implemented.  The  specific  patten  classifier  used  in 
the  research  is  described  in  Chapter  5. 


CHAPTER  111 


DATA  ACQUISITION  AND  FEATURE  CALCULATION 

3.1  Introduction 

A  decision  as  to  what  to  measure  must  be  made  early  in  an  applied 
pattern  recognition  program.  The  type  of  feature  utilized  will,  of  necessity, 
vary  depending  upon  the  particular  application.  For  the  case  of  cough 
categorization,  the  physiological,  model  of  the  cough  reflex  would  be 
expected  to  indicate  candidate  featuias .  This  physiological  model  is 
presented  in  this  chapter  of  this  report  and  is  followed  by  a  description 
of  data  acquisition  methods  and  an  outline  of  the  algorithms  used  for 
feature  calculation. 

3.1  Physiological  Considerations 

3,1.1  The  Respiratory  Mechanism 

/dr  enters  the  respiratory  system  vi.  the  oral  or  nasal  cavities 
which  open  into  the  pharynx.  The  pharynx  separates  into  the  trachea  and 
the  esophagus  directly  above  the  larynx.  At  the  point  of  division,  food  is 
separated  from  an,  food  being  diverted  to  the  esophagus  by  the  closure  of 
the  epiglottis,  a  flap  which  closes  over  the  opening  of  the  trachea  when 
food  touches  the  pharynx.  When  the  epiglottis  is  not  blocking  the  open¬ 
ing  to  the  trachea  the  lower  respiratory  tract  presents  less  resistance  to 


the  flow  of  air  than  the  path  through  the  esophagus.  Ventilation  is  thus 
provided  to  the  lungs. 

The  larynx  is  located  at  the  top  of  the  trachea .  The  vocal  cords 
are  the  portion  of  the  larynx  that  produce  sound.  These  cords  are  tv.-o 
small  vanes  located  on  either  side  of  the  ait  passageway.  The  vanes 
meat  at  an  angle  at  the  back  of  the  larynx.  As  the  muscles  of  the 
larynx  are1  contracted,  closure  of  the  vanes  starts  at  the  point  of  in¬ 
tersection  and  works  its  way  to  the  base  of  the  triangle  they  form.  During 
phonatlon  they  essentially  close  oil  the  trachea.  Air  forced  through  the 
closed  vanes  sets  up  a  lateral  vibration  which  modulates  the  air  stream 
at  audible  frequencies.  The  frequency  of  vibration  is  determined  by  the 
degree  of  contraction  of  muscles  in  the  larynx. 

The  trachea  continues  below  the  larynx  to  a  point  beneath  the  top 
of  the  lungs  where  it  branches  into  the  bronchial  system.  The  bronchi 
are  tubes  of  varying  diameter,  each  bronchus  branching  into  a  network 
of  smaller  bronchi,  thereby  forming  the  "bronchial  tree".  The  smallest 
of  the  bronchi  branch  into  the  alveolar  ducts  which  connect  to  the  alveolar 
sacs  via  atria.  The  alveolai  sacs  are  encased  in  the  pulmonary  membrane, 
which  has  a  thickness  of  from  1  to  4  microns  (several  times  less  than  the 
thickness  of  a  rod  blood  cell).  It  is  in  the  alveolar  saos  Lira L  the  oxygen- 
carbon  dioxide  exchange  takes  place  between  tire  blood  and  the  an  ir  the 
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The  lungs  are  enclosed  in  the  thoracic  cage.  A  negative  pressure 
exists  in  the  pleural  cavity  (a  potential  space  between  the  lungs  and  chest 
wall)  with  respect  to  the  interior  of  the  lungs.  An  expansion  of  the  thoracic 
cage  therefore  expands  the  lungs,  while  a  contraction  of  the  cage  produces 
expiration  . 

The  major  muscles  which  enLer  into  inspiration  are  the  diaphragm, 
the  external  intercostals,  and  a  number  of  small  muscles  in  the  neck.  The 
downward  movement  of  trie  diaphragm  pulls  the  bottom  of  the  pleural  cavity 
downward  (thereby  elongating  it)  while  the  external  intercostals  and  neck 
muscles  lift  the  front  of  the  cage,  causing  the  ribs  to  angulate  forward 
(increasing  the  thickness  of  the  cage). 

The  major  muscles  of  expiration  arc  the  abdominals  and,  to  a  lessor 
extent,  the  internal  intercostals.  The  abdominal  muscles  pull  downward  on 
the  chest  cage  (decreasing  the  thoracic  thickness)  arid  force  the  abdominal 
contents  upward  against  the  diaphragm  (decreasing  the  longitudinal  dimen¬ 
sion  of  the  pleural  cavity).  The  internal  intercostals  aid  slightly  in  expira¬ 
tion  by  pulling  trio  ribs  downward  (decreasing  the  thickness  of  the  chest). 

3.1.2  The  Cough  Reflex 

The  cough  reflex  provides  a  means  for  the  body  to  clear  its  airways. 
It  is  triggered  by  an  irritant  touching  the  surface  of  the  glolLis,  the  trachea, 
or  a  bronchus,  Tire  respiratory  muscles  first  contract  very  strongly  building 
up  high  pressure  in  the  lungs  while  simultaneously,  the  epiglottis  blocks 
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the  trachea  and  the  vocal  cords  clamp  tightly  closed.  These  impediments 
to  air  flow  are  suddenly  removed.  This  allows  the  pressurized  air  to  flow 
out  of  the  lungs  at  high  velocities  carrying  unwanted  particles  with  it. 

Air  flow  and  pressure  measurements  have  been  made  during  coughs 

on  normal  subjects  and  patients  suffering  from  various  respiratory  diseases. 

14 

A  study  by  Whittenberger  and  Mead  indicated  that  peak  interthoracic  pres¬ 
sure  relative  to  atmospheric  pressure  ranged  between  90  and  152  nun  hg  on 
the  six  subjects  used  (three  normal,  one  asthmatic,  two  with  emphysema 
of  long  standing).  Maximum  flow  rates  ranged  between  400  and  700  liters 
per  minute  for  the  normals  and  25  to  150  liters  par  minute  for  the  patients 
with  emphysema  .  The  total  volume  expired  during  the  first  0.2  sec  of  the 
cough  ranged  from  0.15  liter  for  one  of  the  patients  with  emphysema  to 
1.19  liters  for  one  of  the  normals.  The  pressure  was  sustained  fora 
considerably  longer  period  of  time  for  the  diseased  subjects  than  for  the 
normal,  indicating  a  higher  impedance  to  flow  and  decreased  efficiency 
of  the  cough . 

Other  studies  have  rodiologically  examined  the  outline  of  the 
trachea  and  bronchi  during  cough.  There  seems  to  be  no  question  as  to 
whether  or  not  these  passages  contract  during  the  cough,  but  precisely 
what  mechanism  is  involved  is  in  doubt.  It  lias  been  shown  experimentally 
that  the  airway  passages  contract  even  during  normal  expiration. 
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^iRienzo  postulated  a  true  peristaltic  action  which  aided  in  the 
explusion  of  the  irritant.  Other  researchers  disagree,  stating  that  the 
contraction,  although  variable  along  the  length  of  the  airways,  is  not  a 
coordinated  peristaltic  action  at  all,  but  suggest  that  the  contraction  is 
simply  a  means  for  increasing  the  laminar  air  velocity  to  that  required 
for  efficient  cleansing  of  the  airways. 

There  is  some  indication  that  the  increased  resistance  to  flow 
is  highly  localized.  Persons  suffering  from  emphysema,  asthma,  and 
bronchitis  are  unable  to  achieve  these  high  velocities.  A  study  was 
performed  in  which  a  bronchiodilutor  was  administered  to  patient..,  suf¬ 
fering  from  bronchitis  ,  others  having  emphysema  ,  and  a  group  of  normals 
for  control. 1  It  was  found  that  the  air  velocity  increased  for  those  with 
bronchitis,  was  unchanged  for  those  with  emphysema,  and  was  un¬ 
changed  for  the  normals.  This  gives  some  indication  as  to  the  general 
origin  of  the  resistance  to  flow  in  the  two  disease  classes. 

3.2  Data  Acquisition 
3.2.1  Recording 

The  data  with  which  this  paper  is  concerned  were  obtained 
in  the  form  of  analog  tape  recordings.  The  recordings  used  fall  into  two 
categories  --  those  recorded  directly  from  hospital  rooms  at  Woodlawn 
Hospital  in  Dallas,  Texas  and  recordings  of  forced  coughs  and  simulated 

_ »  i  C - *.  ~ 

CJ  1  Liiao  L  D  , 
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The  hospital  recordings  were  obtained  with  the  following  equip¬ 
ment  arrangement.  Microphones  Tere  placed  in  selected  hospital  rooms, 
primarily  in  closets.  The  signals  from  the  microphones  were  carried, 
via  long  cables,  to  recording  apparatus  in  the  basement  of  the  building. 
The  signals  from  the  microphones  were  recorded  on  a  continuous  tape 
loop.  The  amplified  signal  from  the  microphones  were  operated  on  by 
a  voice  controlled  relay  connected  in  parallel  with  the  continuously 
operated  tape  recorder.  A  filter  was  adjusted  empiricially  so  that 
the  voice  controlled  relay  threshold  was  not  exceeded  by  all  artifacts, 
but  it  was  not  possible  to  efficiently  separate  all  coughs  and  artifacts 
by  this  adjustment.  The  voice  controlled  relay  actuated  a  second  tape 
recorder  which  was  operated  in  a  sLart--stop  mode.  The  continuous 
loop  recorder  output  provided  the  input  for  the  start---stop  recorder 
and  allowed  sufficient  delay  for  the  second  recorder  Lo  come  up  to 
speed  after  having  been  actuated  by  the  voice  controlled  relay.  Re¬ 
cording  speed  of  that  start--.stop  recorder  was  3  3/4  ips  . 

Recordings  were  made  by  the  above  described  equipment  mostly 
at  night*  which  is  a  period  of  light  hospital  activity,  to  avoid  recording 
the  numerous  acoustical  artifacts  which  occui  during  the  normal  daylight 
routine.  The  equipment  was  unattended  for  the  majority  of  the  time  that 
recording  took  place. 
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The  recordings  so  obtained  are  typically  characterized  by  high 
background  noise  (with  respect  to  acoustical  events  occurring  in  the 
hospital  rooms)  and  widely  varying  recording  levels.  Fidelity  was  not 
at  the  most  desirable  level  because  of  the  low  recording  speeds  and  the 
ru-recording .  Tor  the  most  part,  however,  coughs  and  artifacts  could  be 
distinguished  by  ear. 

Recor  dings  of  simulated  coughs  and  artifacts  were  made  both  irr 
the  laboratories  at  Woodlawn  Hospital  and  at  The  University  of  Texas 
at  Austin.  Recording  speed  was  7  1/2  ips  and  recording  levels  were 
monitored.  Coughs  were  forced  coughs  of  normal  subjects.  Some  arti¬ 
facts,  such  as  conversation  and  doors  closing,  were  recorded  directly. 
Others  were  obtained  from  commercial  sound-effects  recordings. 

3.2.2  Digitization  and  Segmentation 

Parly  in  the  research  program  it  was  necessary  to  make  a 
decision  as  to  ihe  basic  philosophy  to  be  loliowed  in  data  reduction. 

Two  alternatives  were  apparent.  The  first  was  to  build  special  purpose 
circuitry  to  pic -process  the  recorded  audio  signals;  the  second  was  to 
digitize  the  audio  data  directly  and  perform  all  data  reduction  on  a 
general  purpose  computer.  The  .second  alternative  was  chosen  in  the 
interests  of  flexibility. 

Two  computer  facilities  were  utilized.  Tire  electrical  Engineering 
Department  at  lire  University  maintains  a  Scientific  Data  Systems  9  30 
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computer  which  includes  as  peripheral  equipment  analog -to -digital  and 
digital -to-analog  converters.  The  University  computation  center  main¬ 
tains  a  Control  Data  Corporation  6600  computer.  The  SDS  930  is  operated 
on  an  open  shop  basis  while  the  CDC  6600  computer  is  operated  on  a  closed 
shop  basis.  Data  digitization  was  performed  by  the  SDS  9  30  computer 
while  the  majority  of  the  computations  were  accomplished  by  the  CDC 
6600  computer. 

Prior  to  digitization  of  the  data,  spectrograms  were  made  of  a 
representative  sample  of  the  analog  signals.  It  was  found  that  the 
highest  frequency  of  significant  magnitude  was  in  the  vicinity  of  6  Khz. 

The  Nyquist  rate  is  therefore  approximately  12  K  samples/sec.  In 
practice  it  would  be  desirable  to  have  a  sampling  rate  somewhat  in 
excess  of  this  value.  A  program  was  written  for  the  SDS  9  30  computer 
which  allowed  digitization  at  a  sampling  rate  of  10,500  samples/sec. 

The  A/D  converter  performs  a  12  bit  conversion  with  a  maximum 
conversion  rate  in  excess  of  30  K  conversions/scc .  Conversion  accuracy 
is  specified  by  the  manufacturer  asj_  the  least  significant  bit.  Full  scale 
input  used  was  +  10v  to  yield  a  quantization  error  of  approximately  10  mv. 
The  central  processor  memory  consists  of  8,I9?.--24  bit  words.  The 
nominal  length  of  a  signal  to  be  digitized  is  two  seconds.  At  the  16,500 
conversions/sec  rate  used,  the  available  memory  is  not  adequate  to  buffer 


the  digitized  data. 
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The  magnetic  tape  units  which  are  included  in  the  peripheral 
equipment  may  be  operated  in  an  interlaced  mode  of  operation,  however. 
When  operated  in  this  fashion  access  to  the  central  processor  is  required 
only  when  data  is  required  from  the  memory.  The  interlaced  mode  utilizes  a 
priority  interrupt  system.  One  of  the  priority  interrupts  is  available  for 
program  use  in  a  real-time  environment.  This  interrupt  was  used  to 
signal  the  beginning  of  a  conversion. 

The  A/D  conversion  program  controls  time  of  sampling  to  within 
-  0,  +  3.5  microseconds.  The  digital  data  is  formatted  and  buffered 
in  the  central  processor  memory  while  previous'y  converted  data  is 
simultaneously  written  on  digital  magnetic  tape.  The  conversion  rate 
is  limited  by  the  speed  of  the  tape  transport.  Although  a  higher  con¬ 
version  rate  could  be  accomplished  at  an  800  bpi  data  densiLy,  55G 
bpi  recording  density  was  used  in  the  interests  of  greater  reliability. 

A  digital -to-analog  conversion  sub-routine  was  included  in  the 
program.  Use  of  the  interlace  feature  was  made  to  allow  simultaneous 
D/A  conversion  and  reading  of  the  digital  tape.  It  was  found  that  the 
D/A  output  could  be  directly  connected  to  a  speaker  to  yield  an  audible 
output  with  sufficient  fidelity  for  monitoring  purposes.  Additional  D/A 
features  were  incorporated  to  output  synchronization  and  calibration 
signals  for  oscilloscope  or  oscillograph  monitoring. 
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The  tape  recorder  output  was  connected  to  the  input  of  an  adjust¬ 
able  filter  with  attenuation  characteristics  of  24  db  per  octave.  The  break 
frequency  was  set  at  4  kHz.  The  output  of  the  filter  was  connected  to  the 
computer  multiplexer  --  A/D  converter  input. 

As  previously  stated,  conversions  were  initiated  by  a  priority 
interrupt.  A  pulse  generator  provided  the.  input  signal  to  the  interrupt. 

The  pulse  repetition  rate  was  16,500  pps ,  j_  1  pps.  The  output  of  the 
pulse  generator  was  continously  monitored  by  a  digital  counter. 

Conversion  was  initiated  and  ended  by  computer  console  sense 
switch  operation.  After  an  A,/D  conversion  was  completed  confirmation 
of  the  digitized  signal  was  made  by  utilization  of  the  D/A  conversion 
subroutine . 

The  digital  data  tape  which  was  the  product  of  the  program  was 
formatted  with  1  ,000  --  12  bit  conversions  per  record.  The  binary  re¬ 
presentation  was  integer  two's  complement.  An  identification  record 
followed  data  records  within  a  file,  a  file  consisting  of  the  data  con¬ 
verted  during  an  A./D  conversion  cycle.  Adjacent  files  were  separated 
by  End  of  File  marks  with  the  last  file  on  ^he  tape  being  followed  by 
double  End  of  File  marks. 

The  CDC  6600  computer's  binary  representation  of  integer  data 
is  in  one's  complement,  sixty  bits  per  word.  The  C600  was  programmed 


to  reformat  the  data. 
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Varying  recording  levels  in  the  original  analog  data  required 
that  the  data  be  normalized  with  respect  to  amplitude.  This  was  ac* 
complished  by  arbitrarily  setting  the  datum  point  in  a  file  with  the 
greatest  magnitude  to  a  floating  point  magnitude  of  10,0  and  scaling 
the  remainder  of  the  data  points  in  the  file  accordingly.  Additionally, 
efficient  computation  by  subsequent  programs  required  that  the  files  be 
reformatted  so  that  the  identification  record  proceeded  the  data  in  the 
file.  Provision  was  also  made  to  ensure  that  the  mean  of  the  data  was 
0.  The  normalized  data  was  written  in  binary  format  on  magnetic  tape. 

Due  to  the  manual  start  -  -  stop  digitizing  technique  used  and 
the  nature  of  the  original  analog  data,  files  contained  segements  of  low 
level  data  (backgound  noise).  It  was  therefore  necessary  to  locate  seg¬ 
ments  of  the  files  which  contained  usable  data.  An  algorithm  subse¬ 
quently  used  to  calculate  the  Fourier  series  representation  of  the  time 
varying  signal  required  that  segments  of  data  used  in  the  calculation 
contain  a  number  of  data  points  exactly  divisible  by  a  power  of  two 
(this  algorithm  is  described  in  a  later  section  of  the  dissertation). 

The  divisor  chosen  was  1024  points.  A  file  was  searched  until  a  value 
exceeding  a  specified  threshold  L,  was  encountered.  The  conversion 
number  of  that  value  was  assigned  as  the  start  of  a  segment.  Subsequent 
data  points  were  assigned  to  the  segment  until  no  values  exceeding  L 
were  found  for  a  pre-spocifi ed  time  interval,  t  .  The  number  of  conversions 
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corresponding  to  t  ,  were  subtracted  from  the  total  number  of  points  assigned 
d 

to  the  segment  to  obtain  the  length  of  the  segment.  If  the  segment  so 

located  was  shorter  than  a  specified  time  limit,  t  ,  tire  segment  was 

r 

rejected.  After  segmentation  of  a  file  was  completed,  segment  lengths 
were  adjusted  to  be  an  integral  multiple  oi  1024  points.  The  values  L, 

t  ,  and  t  were  determined  empirically .  The  following  values  were 

r  d 

found  to  give  satisfactory  results: 

L  -  2.0 

t  =  t  =  1024/10500  sec  1/1  6  sec 
d  r 

The  normalizing  factor  for  eac ..  .  was  recorded  for  future  use  in 

calculation  oi  the  significant  noise  ievel  . 

Alter  segmentation  was  accomplished  on  the  CDC  0600,  the 
original  data  was  scanned  on  the  SDS  930  by  the  D/A  conversion  sub- 
rountine.  Class  identification  of  the  segments  and  1024  point  subsegments 
was  confirmed  by  listening  to  the  D/A  converted  data. 

Summarizing  the  digitization  and  segmentation  programs,  analog 
data  was  digitized  on  the  SDS  930  at  a  conversion  rate  oi  1G500  conver- 
slons/seo.  The  data  obtained  was  normalized  with  respect  to  amplitude 
and  subsegmented  on  the  CDC  6600  computer.  Confirmation  of  segmentation 
and  class  identification  (cough  or  artifact)  was  then  made  by  use  of  the  SDS 
930  D/A  conversion  subroutine. 
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After  the  completion  of  the  above  process  the  digitized  data  was 
on  magnetic  tape  in  a  format  compatible  with  computation  in  the  CDC  6 v G 0 . 
Punched  cards  were  included  in  the  output  Horn  the  0600  ;  .egram  on  which 
were  entered  segment  and  class  (cough  or  artifact)  Identification .  Alter 
the  original  digitized  data  was  scanned  on  the  6DS  930  to  confirm  class 
identification  of  the  subsegments,  the  punched  cards  were  altered  if 
original  class  identification  was  enormous  ,'z  section  of  background 
noise  identified  as  a  cough) .  The  punched  earns  were  then  used  as 
input  data,  along  with  the  6600  formatted  tape,  for  subsequent  programs 
on  the  CDC  6600.  Data  not  included  within  one  of  the  subsegments  was 
not  considered  in  subsequent  calculations . 

3.3  I’eaturc  Lxtraction 

The  measures  calculated  are  described  in  sections  3.3.1  tlmough 
3.3.4  of  this  report.  They  fall  into  four  broad  categories: 

1)  those  concerned  wiLh  the  magnitude  of  tire  normalized 
amplitude  of  the  signal. 

Z)  those  which  represent  the  form  of  the  envelope  of  the  signal. 

3)  zero  crossing  representations  . 

4)  spectral  analysis  measures. 

3.3-1  Amplitude  Density  Approximation  feature. 

An  examination  of  the  waveform  of  the  recorded  signals  indicated 
that  an  analysis  of  the  shape  of  the  amplitude  uisUibutrun  of  the  normalized 
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signals  could  offer  possibilities  as  a  measure. 

A  102  4  point,  subsegemont.  of  data  was  operated  on  to  obtain  an 
est.mate  of  the  probability  density  function  of  the  amplitude  of  the  signal. 
Thirty-one  equi-spacea  limits  were  used  to  yield  thirty-two  increments. 

The  increment  limits  are  as  shown  in  Table  3-1  .  It  is  noLed  that  normaliza¬ 
tion  accomplished  during  reformatting  was  with  respect  to  the  point  with 
maximum  magnitude  within  s.  recorded  file  and  that  several  segments  and 
subsegments  were  incLuded  within  this  file.  The  mean  value  of  all  points 
within  a  file  was  zero,  but  such  was  not  necessarily  the  case  within  a 
subsegment . 

The  algorithm  used  to  implement  the  amplitude  density  approximation 
is  given  below.  The  in.rjmcnt  address  calculations  take  'he  lari',  of  a  tree 
where  the  iterative  decisions  described  determine  the  branch  chosen.  The 
tree  terminates  with  a  total  number  of  brant  hes  equal  to  the  number  of 
classification  intevals. 


Define: 


f_\ 

I  ~  value  of  ilh  sample  in  a  subsegment 

Number  of  intcivals  used  =  2M^  ^ 

D  =  interval  width  (difference  of  interval  limits) 

V  =  2MD 
A 

L  =  interval  number;  highest  amplitude  value,  lowest 

,  M+  1 


number;  L  -  1 ,  2 


2 
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Table  3-1 
Increment  Limits 

Amplitude  Density  and  Envelope  Amplitude  Density 

Features 


Increment 

No. 

Lower 

Limit 

Upper 

Limit 

Increment 

No. 

Lower 

Limit 

Upper 

Limit 

1 

-9.375 

J  7 

0.000 

0 . 625 

2 

-9 . 375 

-8.750 

18 

0.625 

1  .250 

3 

-8.750 

-8.125 

19 

1 .250 

1.875 

4 

-8.125 

-7.500 

20 

1.875 

2 .500 

C 

-7.500 

-6.875 

21 

2.500 

3.125 

6 

-6.875 

-6.250 

2  2 

3.  125 

3.750 

7 

-6.250 

-5 .62  5 

23 

3.750 

4.375 

8 

-5. G25 

-5.000 

24 

4.375 

5.000 

9 

-5.000 

-4.375 

2  5 

5.000 

5.625 

10 

-4.375 

-3.750 

26 

5.625 

6.250 

]  1 

-3.750 

-3 .125 

27 

6.250 

6 . 875 

12 

-3.125 

-2 . 500 

28 

6.875 

7 . 500 

13 

-2 . 500 

-1 . 875 

29 

7 . 500 

8. 125 

1  4 

-1 . 875 

-1 .250 

30 

0. 125 

0.750 

1 5 

-1 .2  50 

-0 . 62  5 

31 

8.750 

9.37  5 

16 

-0.025 

O.OOU 

32 

9 . 375 

31 

A 

X  =  dummy  variable  used  in  calculation  of  interval 
address  of  the  jth  sample  point 

The  following  iterative  algorithm  identifies  the  interval  address  of  a  sample 
with  a  total  of  M  +  1  tests.  Parenthesized  superscripts  denote  iteration 
number: 

x  <0)  =  r  ;  L(0)^l 
J  1  3 


If: 


x}1  -  V/21  <  0  ;  L  ^  =  I.  ^-1'  +  2M  1 

J  3  3 


x(l)  =  x(i_1) 

3  3 

>  0  ;  L  (l)  - 
—  3  3 

X.(i)  =X  (i_1)  -  v/21 
3  3 


1  =  1,  2 . M 

(M+l)  M+l 

F.  <0  ;  L.  -  2  -  L. 

J  3  3 

.  n  T  (M+l)  _  (M) 

~  ''  3  _L3 


It  is  noted  that  this  aiyoiuhm  is  most  efficient  if  the  F/s  are 
uniformly  distributed  between  +  V. 

The  CDC  GG00  is  capable  of  calculating  several  arthmetic  operations 
simultaneously  providing  that  the  calculations  are  indepondcnl.  When  a 
calculation  is  dependent  upon  the  outcome  of  a  test,  such  is  not  the  case. 
Minimization  of  the  number  of  testy  is  therefore  roughly  equivalent  to 
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minimization  of  calculation  time.  Table  lookups  were  employed  to  obtain 
21  and  V/21  in  the  above  algorithm  so  that  these  time  consuming  calculations 
were  not  made  during  every  iteration. 

After  the  above  algorithm  operated  on  all  data  points  in  a  subsegment, 
the  frequency  of  occurrence  values  were  normalized  in  order  to  estimate  the 
amplitude  density  functions  by  dividing  each  interval  count  by  the  total 
number  of  points  classified.  For  convenience  in  later  calculations,  the 
array  storing  the  normalized  values  was  then  inverted  so  that  the  lowest 
numbered  interval  corresponded  to  the  most  negative  limit. 

3.3.2  Envelope  Descriptive  Features 

A  preliminary  study  of  the  waveform  of  the  recorded  coughs  and 
artifacts  indicated  that  the  set  of  candidate  features  should  include 
envelope  descriptive  measures. 

The  cough  signal  started  with  a  large  amplitude  and  died  off 
quasi-exponentially ,  as  would  tie  expected  from  the  physiological  model. 

This  was  not  the  case  for  the  majority  of  the  waveforms  of  recorded 
artifacts.  The  two  envelope  dependent  features  which  were  calculated 
were  an  estimation  of  the  probability  density  of  the  envelope  amplitude 
and  an  estimation  of  the  probability  density  of  the  slope  of  the  envelope. 

3 . 3 . 2  .  1  Envelope  Amplitude  Density  Feature 

In  order  to  estimate  the  probability  density  function  of  the  ampli¬ 
tude  of  the  envelope,  a  sampled  representation  of  the  envelope  was  required. 
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Initially  the  peak  values  of  the  signal,  which  were  points  on  the 
envelope,  were  obtained  by  applying  Sterling's  interpolation  approximation 
from  numerical  analysis.  The  time  of  maxima  oi  minima  were  computed  by 
differentiation  of  Sterling's  formula  and  then  solving  for  tha  time  when 
the  derivative  of  the  approximation  is  zero.  A  re-application  of  Sterling's 
formulation  yielded  the  interpolated  value  of  the  amplitude  of  Lite  signal 
at  the  time  of  occurrence  of  the  extrema. 

Sterling's  approximation  attempts  to  fit  a  polynomial  to  the  dis¬ 
crete  poinLs  in  the  neighborhood  of  the  point  of  interest.  Having  evaluated 
the  coefficients  of  the  polynomial,  one  solves  for  the  value  at  the  point  of 
interest.  A  comparison  of  the  values  obtained  by  this  approach  to  that 
obtained  by  noting  the  time  of  occurrence  and  the  amplitude  of  the  peak 
sampled  points  indicated  that  little  difference  existed.  In  the  interests 
of  computational  efficiency,  the  algorithm  used  for  extraction  of  a  sampled 
representation  of  the  envelope  used  this  latter  approach. 

A  1024  point  subsegment  was  scanned  to  locate  the  maximum  and 
minimum  points  .  The  time  of  occurre  ice  and  amplitude  of  the  extreme 
were  stored  in  central  memory.  The  algorithm  utilized  to  obtain  the  peak 
points  and  times  of  occurrence  is  described  below.  It  will  be  noted  that 
the  maxima  are  positive  and  the  minima  negative. 

A 

T  =  magnitude  of  maximum  peak  to  peak  noise 


Define: 
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F.  =  value  of  jth  data  point  (relative  to  start  of 
^  subsegment) 


N  -  number  ol  positive  maxima 
P 

A  . 

N  =  number  of  negative  minima 
n 

A 

D  =  dummy  variable 
A  ,  ,  .  ,th 

F  -  amplitude  of  l  positive  maximum 
Pf 

E  =  amplitude  of  n*" 1  negative  minimum 
nm 

M  ,=  sample  number  of  £th  positive  maximum  re- 
1  lative  to  start  of  subsegment  (proportional 
to  time  of  occurrence) 

Iv)  ^sample  number  of  rr/^  minimum  relative  to 
nrn  start  of  subsegment 


The  following  algorithm  identifies  and  stores  the  quantities  E  , 


E  ,  M  and  M 


nrn  pc  nm  p 

superscripts  denote  iteration  number: 


The  quantities  N  and  N  are  computed .  Parenthesized 


D(0)£  ,  l,(o)4o-  m<0)  -0.  N  (o!=0.  N  (o)  =  0 

P  n 


Define  the  follov/ing  events: 


A0)  =(?.:  D(l_1V  0,  |r.+  j  -IV  |  >T,  Syn  (F.+  ]-F.) 
¥■  Sgn  D(l_1)i 


y- 

1 

1 

V 

o 

HW£{- 

B(i)  *  G(i) 

n  ,,W 

II  *  J 

C(^G(i; 

'  n  ii(l) 
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Define  the  indicator  functions 

(i) 


I 


A 


i)  _  f  0  1 

ll  i 


it  UP.™ 

f  5  e  Aw 


B 


(i)  =  (o  if  ?  ^b;; 

ll  if 


5  e  B 


(0 

(0 


“  >B(° 


(i)  =  ( 0  if  ?  f[  C' 
C  ll  if  £  e  C 

-  1  then  m(0=  m(‘+1) 

M<‘>  =  . 
n  m 


a) 


a) 


ntn 


=  F. 


If  I  (i)  I  0) 
A  C 


Du)  =  +  1 

N  (l)  -  N  (l_1)  +  1 
n  n 

-  1  then  -t(l)  =  4  1 

..  (i) 

M  ,  =  i 

pt. 

E  (l)  =  F 
p-h  l 

-  -1 

N  (1)  =  N  (l_1,+  ] 

1-  P 


Otherwise  quantities  are  unchanged  from  the  previous  iteration. 

After  the  peak  points  have  been  obtained,  the  L  1  s  and  E  's  are 

P  n 

used  to  estimate  the  probability  density  of  the  envelope  amplitude  by 
implementation  of  the  algorithm  described  in  Section  3.3.1.  Thirty-two 
increments  ware  utilized  in  the  approximation.  Limit  values  were  as  given 


in  Table  3  -1  . 
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3. 3. 2. 2  Envelope  Difference  Density  Feature 

The  slope  of  the  envelope  was  approximated  by  dividing  the  dif¬ 
ference  in  amplitude  of  adjacent  extreme  points  by  the  number  of  sampling 
intervals  between  these  points.  All  quantities  required  for  these  calcula¬ 
tions  were  obtained  by  use  of  the  algorithm  described  in  Section  3. 3. 2.1. 

The  maximum  and  minimum  points  were  operated  on  separately. 

Prior  to  calculation  of  the  envelope  differences  for  the  negative  envelope, 
these  minima  were  replaced  by  their  negatives  (to  yield  all  positive  points) 
After  calculation  of  the  differences  ,an  estimate  of  their  probability 
density  was  made  by  application  of  the  algorithm  outlined  in  Section  3.3.1. 
Sixty-four  intervals  were  used  in  the  approximation  with  limits  as  shown  in 
Table  3-2  . 

3.3.3  Zero- Crossing  Features 

A  measure  which  is  frequently  used  because  of  its  computational 
simplicily  is  a  count  of  zero  crossings  of  a  signal  of  a  signal  during  a 
specified  tinis  intorvsl.  Altsmstivsly sursmont  of  ths  periods  Id  g  t w 3 o n 
zero  crossings  would  contain  moia  complete  information. 

The  algorithm  that  was  implemented  calculated  the  time  interval 
(to  the  nearest  sample  period)  between  zero  crossings  as  a  first  step.  A 
dead  band  (centered  at  zero)  which  exceeded  the  maximum  noise  level 
was  assigned  prior  to  implementation  of  the  algorithm.  It  was  required 
that  the  signal  pass  through  the  dead  band  before  a  zero  crossing  was 
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Table  3-2 
Increment  Limits 
Envelope  Difference 
Feature 

Incre-  Incre-  lucre-  Incre- 


ment 

No. 

Upper 

Limit 

ment 

No. 

Upper 

Limit 

ment 

No. 

Upper 

Limit 

ment 

No . 

Upper 

Limit 

1 

-1 . 

,695 

17 

-0 . 

820 

33 

0. 

.055 

49 

0.930 

2 

-1 . 

,641 

18 

-0. 

766 

34 

0 . 

.109 

50 

0.984 

3 

-1 . 

,586 

19 

-0. 

711 

35 

0, 

,164 

51 

1  .039 

4 

-1  . 

,531 

20 

-G . 

656 

36 

0. 

.219 

52 

1.094 

5 

-1 , 

.477 

21 

-0. 

602 

37 

0, 

.273 

53 

1.148 

6 

-1 . 

.422 

22 

-0  . 

547 

38 

0, 

.328 

54 

1.203 

7 

-1 . 

.367 

23 

-0. 

493 

39 

0 

.383 

55 

1.250 

8 

-1 , 

.313 

24 

-o. 

438 

40 

0 

.438 

56 

1.313 

9 

-1  , 

,258 

25 

-0. 

383 

41 

0 

.  493 

57 

1.367 

10 

-1 , 

,203 

26 

-0. 

328 

42 

0 

.547 

58 

1.422 

11 

-1 . 

.148 

27 

-0. 

27  3 

43 

0 

.  602 

59 

1.477 

12 

-1  , 

.094 

28 

-0  . 

219 

44 

0 

.  656 

60 

1..  531 

13 

-1 , 

.039 

29 

-0. 

164 

45 

0 

.  711 

61 

1  .  586 

14 

-0, 

,984 

30 

-0. 

109 

46 

0 

.766 

62 

1 . 641 

15 

-0, 

.930 

31 

-0, 

055 

47 

0 

.  820 

63 

1.695 

1G 

-0 

.  875 

32 

0. 

000 

40 

0 

.  87  5 
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tabulated , 

Memory  locations  corresponding  to  logarithmically  spaced  bands, 
limits  of  which  are  given  in  Table  3-3,  ware  incremented  in  accordance 
with  the  number  of  sampling  increments  between  zero  crossings.  After 
a  1024  point  subsegment  of  data  had  been  operated  on,  the  values  for  the 
zero  crossing  period  counts  were  normalized  by  dividing  the  number  of 
tabulated  crossings  per  interval  by  the  total  number  of  zero  crossings 
for  the  102  4  point  subsegment  to  yield  an  estimate  of  the  probability 
density  of  the  zero  crossing  periods.  Additionally,  the  average  zero 
crossing  frequency  was  obtained  from  the  total  number  of  zero  crossings 
tabulated . 

3.3.4  Spectral  Features 

As  previously  noted,  spectrograms  of  selected  recorded  audio 
signals  were  made  and  examined  for  characteristics  which  were  unique 
to  the  audible  cough  as  contrasted  with  the  artifacts.  It  was  expected 
that  resonances  caused  by  the  various  cavities  in  the  vocal  tract  could 
possibly  give-  rise  to  broad  frequency  peaks  analogous  to  formants  on- 

g 

countered  in  speech  recognition  studies.  Formant  like  structures  were 
not  apparent,  however. 

Although  formants  were  not  present,  information  would  still  be 
expected  to  be  contained  in  a  frequency  domain  analysis  of  the  signal. 
Several  approaches  to  obtaining  this  analysis  were  apparent.  A  Fourier 
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i  i 


Band  No. 


1 

2 

3 

4 

5 
€ 


7 


0 

9 

1  C 


Table  3-3 
Band  Limits 
Zero  Crossing  Period 
Feature 

Lower  Frequency*  Upper  Frequency* 

_ Limit _  _ Limit 


0 

16. 

1 1 

16. 

,11 

32  . 

.23 

32. 

.23 

64. 

,  4  5 

64, 

,  45 

i 

A  ■ 

,2  89 

X 

10s 

1  , 

.209 

>; 

10" 

2  , 

,  57  0 

X 

10s 

2  , 

.578 

X 

10s 

5  , 

.  1  56 

X 

10E 

5  , 

.156 

X 

i0" 

1  , 

.031 

X 

10  3 

1 

,031 

X 

1  u:i 

2 

,063 

X 

10“ 

2 

.  003 

X 
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sub¬ 


senes  representation  or  power  spectral  density  estimate  ov  a 
segment  would  be  candidates.  An  alternative  method  of  obtaining 
similai  information  would  be  to  r  •  ■  .1  aigital  band -pass  filtering  on 

the  time  varying  signal  directly  and  to  then  obtain  the  power  output  from 
a  digital  band-pass  filter  over  .segments  oi  time, 

All  oi  the  above  computations  require  a  large  number  of  multiplied  - 
tions.  Techniques  have  been  developed  for  efficient  estimation  ol  the 
power  specLal  density  and  recursive  algor ithms  could  be  implemented 
in  the  case  of  the  digital  filters.  Computation  time  would  still  be  groat, 
however . 

4 

The  advent  of  a  fast  l’ouiioi  algorithm  allowed  direct  calcula¬ 
tion  oi  tin;  compl  ex  l'ouiiei  sei  las  coefficients  oi  subsegments  of  the 
data.  This  algorithm  was  programmed  for  the  CDC  5000  computer.  Since 
the  program  follows  the  published  Cooley -Tuchoy  algorithm  atop  by  step, 
it  will  ii>'t  bj  ue set  mud. 

A  102  1  point  subsegment  ol  data  was  operated  on  to  yield  511 
(excluding  tiro  DC  component)  usable  complex  coefllcients .  The  numbers 
obtained  were  proportional  tu,  lulhei  than  equal  to,  the  I'ouriur  coefficients 
since  divisions  nucu&sut y  to  tuba  Into  account  the  sampling  rate  anti  period 
of  the  subsegment  were  not  perlormud;  sampling  rates  and  subsegment  length 
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arc  identical  for  all  data  processed  and  tha  values  are  required  foi  com¬ 
parative  purposes  only. 

Because  of  the  large  number  of  coefficients  which  resulted  from 
the  computations  outlined  above,  it  was  necessary  to  effect  further  data 
reduction.  This  was  done  by  summing  the  squared  magnitudes  of  the  co¬ 
efficients  over  bands  and  by  extracting  information  concerning  peak 
frequencies  and  amplitude  of  the  magnitude  squared  of  tire  coefficients 
at  these  peak  frequencies. 

3. 3.  4.1  fourier  Band  features 

As  noted  above,  the  squared  magnitudes  of  the  Fourier  coefficients 
were  summed  over  bands.  A  total  oi  nine  logarithmically  spaced  bands 
wore  used.  Band  limits  are  shown  in  Table  3.4. 

3. 3.  4.  2  frequency  I'ual;  features 

The  squared  magnitude  of  the  l'ouiiot  coefficients  were  operated 
on  by  the  envelope  deteiminlng  algorithm  described  in  Section  3.  3.2.1  of 
this  report  to  determine  the  frequencies  at  which  amplitude  peaks 
occurred,  'Jliu.se  maxima  weie  limn  unified  will'  i aspect  to  magnitude  arid 
the  twenty  peaks  with  the  greatest  magnitude  wcie  .selected.  A  total  oi 
eighty  measures  were  extracted  in  the  luim  of  the  magnitude  ol  the 
twenty  highest  peaks,  the  twenty  frequencies  corresponding  to  thong 
peak.!  and  tiie  i  i  equcncJos  cm  either  side  oi  the  peak  f  n*quen.,lus  at 
which  the  squared  magnitude  was  onu-fouiLh  ul  the  peak  value.  In  the 


Table  3-4 


Fourier  Coefficient 


Magnitude  Squared  Bands 
Frequency  Limits 


Upper 

Band 

Limit 

No. 

(Hz) 

1 

16.11 

S' 

A 

48.34 

3 

]  .128  x  ]0R 

4 

2.417  x  10s 

5 

4.995  x  10s 

6 

1.015  x  10a 

7 

2. 040  x  103 

8 

4.109  x  103 

9 

8.234  x  103 
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event  that  less  than  twenty  peaks  were  apparent,  a  value  of  zero  was 
assigned  to  the  measures  for  which  no  peaks  were  found. 

3.3.5  Measure  Summary 

Table  3-5  lists  the  features  which  were  calculated  and  the  measure 
numbers  assigned  to  them. 


3.4  Measure  Selection 

The  algorithms  outlined  in  Section  3.3  of  this  paper  were 
used  to  calculate  a  total  of  228  measures,  A  segment  of  data  contains 
several  subsegments,  each  of  which  is  represented  by  these  228  quantities. 
If  the  segment  consisted  ol  thirty  subsegments  (a  not  unusual  occurrence)  , 
the  pattern  would  have  6840  dimensions.  Such  a  large  dimensionality  is 
unfeasible  for  real  lime  implenentatron  by  special  purpose  circuitry.  The 
fast  access  memory  requirements  £o:  a  general  purpose  computer  implc- 
rrioniat.on  would  be  sizable.  It  was  thercloro  necessary  to  select  a  sub¬ 
set  ol  the  computed  measures  for  use  in  training  and  classification. 

Tire  problem,  then,  is  to  order  the  measures  in  a  manner  which 


reflects  the  maximum  probability  of  correct  classification  or,  equivalently , 
which  minimizes  the  probability  of  v.  ony  classification.  Aflei  order iny  the 
measures  one  would  choose  the  riumner  of  ordered  measures  which  either 


allowable  capacity  of  the  machine. 
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Taole  3 -5 
Table  of  Measures 


Measure 
Nos , 


Feature 


1—32 


Amplitude  density  —  least  values,  lowest 
number 


33  --64  Envelope  amplitude  density  --  least  values, 

lowest  number 

65-128  Envelope  difference  density  --  least  value, 

lowest  number 


129 -138 


Zero  crossing  period  density  --  lowest  fre¬ 
quency  band,  lowest  number 


139 


Average  zero  crossing  frequency 


1  40-148 


Magnitude  squared  Fourier  coefficient 
bands,  lowest  frequency ,  lowest  number 


149-168 


Normalized  frequency  of  Fourier  peaks  -- 
highest  amplitude  peak,  lowest  number 


169-188 


Amplitude  of  Fourier  peaks  corresponding 
to  measures  1 49  —  1 08 


189-208 


Lower  quarter  power  frequency  corresponding 
to  uiuuyu!  us  3  49  -1  68 


209 -228 


Upper  quarter  power  frequency  corr esponding 
to  measures  149-168 


Mil 
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If  the  pattern  is  the  input  to  a  machine  which  depends  upon 
distance  in  some  sense  between  patterns  of  differing  classes  to  effect 
recognition,  one  wishes  to  identify  those  measures  for  which  high  pattern 
densities  in  E^  of  the  opposing  classes  are  least  overlapping.  In  terms  of 
the  joint  probability  density  functions  this  indicates  that  the  chosen  measures 
should  be  those  for  which  the  joint  probability  density  functions,  conditioned 
on  class  membership,  exhibit  the  greatest  difference  in  value. 

If  a  Bayes  machine  is  to  be  implemented,  it  can  bo  shown  that  the 
same  criterion  applies.  Tor  the  two  class  case,  recall  that  the  classifica¬ 
tion  surface  of  the  Bayes  machine  with  a  symmetric  loss  function  is  given 
by: 

p  ( 1 )  p  (X  1 1 )  -  p(2)p(x|2)  =  0  (3-1) 


Define: 


R1  =  [X:P(1)P<X|J)>I'U)P(X|2)} 

r2  =  ^X:p(2)p(x|?.)  *p(l)p(x|l)j 


continuous  probability  density  functions,  the  prob¬ 


ability  of  making  an  eironoous  decision,  P  ,  is  given  by  equation  (3-2). 


r  =  j  •  •  ■  p(2)p(x|2)  dXj  .  .  .  d>;d 

.  A  j  p(J)p(X  |/.)  d  x.  ...  fi  x 
R  “  “  1  d 


(i-2) 


•  ■  ■  I  ]>(1  )p(il |l  )  d>:  ...  d  k 

,,  J  -  1  d 
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-  f  •  •  ■  P  [p(l)p(X|D  -  p(2)p(x|2)]  dXj..  .  dxd 


+  f  p(2)p(X|2)  dx1  ...  dxd 


^  .  .  .  j  Qp  ( Z )  p  (X  1 2 )  -  p(]  )p(X  |  2)]d  x(  ...dxd 


.  .  .  J  P <  1 )  P (X  1 1 )  d  Xj .  .  .d  x 


+  ...  *"p(2)p(X  1 2)  dx  ...  dx 

p  1  U 

2  (3-3) 

OO  c TJ 

-  I  ...J  |  p(l)p(x|i)  -  p(2)p(x|2)  IdXj  .  .  .d  xd 

00  CO 

All 


Minimization  of  P  implies  maximization  of  a ,  defined  in  equation 
e 


(3-4)  giver  below: 


O' 


'  |  p  ( 1 )  p  ( X  !l)  -  p(2)p(x|2)  |  d  x  ...dx 

(3-4) 


If  the  probability  density  functions  aiu  discrete  iathei  than  continu¬ 
ous,  summations  take  the  place  of  integrations. 

Use  of  (3-4)  requires  knowledge  of  the  a  prion  probabilities  of 
occurrence  and  tire  conditional  joint  density  functions  for  sub-sets  of  the 
measures  taken  d  at  u  time.  Should  the  a  priori  probabilities  be  unknown 
and  unestimnble ,  maximum  Likelihood  classification  is  appropriate,  in 


;|ijl|!f|H,j'| . .  .*!  "I>I!!|V,  . . . . . . .  'll  ■'  III:!  .Illi.  Hl|i  l»  l.lMI'l 
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which  case  the  quantity  to  be  maximized  is  given  by  @  ,  defined  in 
equation  (3-5)  given  below: 

CD  er> 

(5  =  j  .  .  .  j  I  P ( X  | L )  -  p(X  1 2)  |  d  x  1 .  .  .d>n 
It  is  seen  that  p  assumes  a  maximum  value  of  2  for  disjoint 


(3-5) 


classes.  Under  the  presumption  of  statistical  indpendence  of  the  measures 

3 

it  has  been  shown  that  minimization  of  equation  (3-5)  is  equivalent  to 
selection  of  the  d  measures  which  maximize  y,  defined  as: 

d 


A  y 

Y  =  ■  s- 


i=l  J 


>(x  |l)  ■■  p(x.  1 2)  |  dx. 


(3-6) 


where  p(x^  |  1)  and  p(x.  |  2)  are  the  marginal  conditional  densities.  Some 
measures  with  which  this  dissertation  is  concerned  undoubtedly  exhibit 
statistical  dependence.  However,  due  to  the  impracticability  of  estimat¬ 
ing  the  joint  probability  functions  and  uncertainly  regarding  a  priori 
probabilities  of  occurrence,  the  relationship  given  in  equation  (3  -6)  was 
used  to  select  the  sub-set  of  measures  upon  which  training  and  recogni¬ 
tion  was  performed. 

The  algorithm  described  in  Section  3.3.1  of  this  paper 
was  applied  to  the  measures  to  estimate  their  marginal  conditional  prob¬ 
ability  density  functions.  The  absolute  values  of  tire  difrei ences  in 
estimated  probabilities  for  the  two  classes  wore  then  summed  and  the 
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resulting  values  ordered  with  respect  to  magnitude.  Results  are  giver, 
in  Chapter  5 . 


CHAPTER  IV 


PATTERN  CLASSIFIER  MODEL 

4.1  General  Considerations 

Prior  to  describing  the  machine  which  was  chosen  to  implement 
the  categorization,  a  brief  review  of  the  desirable  properties  of  the  machine 
is  in  order . 

Since  we  prefer  that  the  model  chosen  be  adaptable  to  real  time 
implementation,  simplicity  from  a  hardware  viewpoint  is  a  consideration. 
The  TLU  previously  described  would  meet  this  requirement,  as  would 
circuitry  which  selects  the  maximum  of  several  signals  which  arc  weighted 
sums  of  the  input  measures.  Networks  which  tealize  a  logical  switching 
circuit  would  also  be  candidates.  Other  possible  implementations  would 
require  the  storage  of  values  in  a  memory  of  some  description.  While 
these  techniques  could  be  implemented  with  relative  case  in  a  general 
purpose  computm  piogiam,  the  real  time  circuitry  would  be  more  complex 
than  would  be  desirable. 

It  has  previously  been  notect  that,  the  use  ol  the  general  13a yc-s 
machine  requires  estimation  (and  subsequent  storage)  of  p  (X  |  i)  and  p(i) 
for  all  X  in  E  ,  which  is  a  formidable  task.  If,  however,  the  measures  to 
be  used  are  binary  valued  and  statistically  independent,  the  Bayes  machine 
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so 


for  the  symmetric  loss  function  assumes  a  particularly  simple  form.  The 

derivation  of  the  discriminant  function  for  this  special  case  is  taken  from 
10 

NUlson 


Recall  that  the  Bayes  discriminant  function  foi  the  symmetric  loss 


function  is  g,(X)  =  p(X/i)p(t) ,  or  equivalently  with  respect  to  the  decisions 

made,  g  (X)  =  In  pfX/i)  +  In  p(i),  i  =  1,  .  .  .  ,  R.  Tor  the  case  of  R  =  2  and 

the  x.  statistically  independent,  we  may  write: 

g(X)  =  £  In  p(x./l)  +  In  p ( 1 )  -  E  In  p  (x./2)  -  In  p(2) 

1=1  1  i=l  3 


Si  .  P(X/U  pfil 

—  E  in  ,  , ,rT  In  ,  . 
i-1  P(x,/2)  p(2) 


(4-1) 


where  the  x,  may  take  on  values  of  only  0  and  1  . 

The  quantities  which  must  be  estimated  have  been  reduced  to  the 
following: 


p(x.  ■=  1/1)  =  p. 

p(x.  =  0/1)  1  -  p. 

p(x.  =  1/2)  =  q, 

l  i 

p(x  ==  0/2)  =  1  -  q 

l  l 


(4-2) 


i-1, 


,  a 


p(l) 

p(2)  =  1  -  p(l) 

Making  use  of  the  fad  that  the  x  may  take  on  values  of  only  0 


and  1 ,  we  may  wt ite: 


p(x./l)  p.  .1  -  p. 

1  n  7rr  =  x .  1  n  —  +  ( 1  -  x , )  1  n  - - - 

p(x,/2)  i  q.  i  1  -  q. 

l  l  l 


Substitution  of  (4-3)  into  (4-2)  yields: 

d  d  3  ~  pi  p(l) 

g(X)  -  T,  x.ln  — — — :  4  L  )  n  - - +  In  '  ,77 

-  1  ■—  1  i  q,(l-  -  Pj)  i  =  l  1  -  q.  1  -  Ptl) 

Equation  t4-4)  is  of  the  form.  g(X)  =  W>X'  with: 


w  ~  1  n 


iv 1  r 
«iu  -  'V 


i  =  1 . d 


w  ,  ,  **  JJ  In  - -  -t  In 


1  1  "  qi 


1  -  p(l) 


Training  of  the  machine  would  then  consist  of  estimating  the  value 

of  p  ,  q  ,  p(l)  and  p(2)  and  substituting  these  estimates  into  the  relation- 
*  1 

ships  given  in  equation  (4-5). 

It  one  docs  not  have  knowledge  of  the  form  of  the  underlying 
probability  density  functions  for  the  x  ,  it  is  not  possible  to  arrive  at  an 
optimal  estimate  of  the  roquJied  parameters  .  One  cun  use,  however,  an 
estimate  which  is  "reasonable  .  " 


Nj  =  Humber  of  patterns  in  a  training  set  belonging  to 
class  1  . 


N  —  Number  of  patterns  in  a  training  set  belonging  to 


class  2 , 
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iii  — w  mM*** 


r  . 


5  3 

as  Inputs  to  the  binary  valued  Bayes  rriachine  previously  described,  we 
should  be  able  to  decrease  Lhe  number  of  wrong  decisions.  Our  model, 
then,  assumes  the  form  of  a  layered  machine,  the  first  layer  being  trained 
by  a  non-probabilistic  algorithm  while  training  for  the  final  layer  is 
probabilistic  in  form.  The  first  layer  will  be  called  the  lust  stage  of  the 
pattern  classifier  and  Lhe  second  layer  will  be  designated  as  the  second 
stage. 

4.2  Non-Probabilistic  Pattern  Classifier  first  Stage 

The  machines  examined  for  use  as  lire  first  stage  wore  themselves 
layered  machines.  Recall  that  a  layered  machine  implements  a  picccwise- 
linear  discriminant  function. 

The  first  banks  oi  a  layered  machine  may  be  thought  of  as  a  device 
which  performs  a  mapping  from  pattern  space  to  on  image  space.  Consider 
the  case  where  poUerns  arc  not  linearly  separable  in  pattern  space.  Il 
can  bo  shown  that  for  a  particular  training  set  a  first  layer  may  be  round 
so  that  the  mapped  patterns  In  decision  space  ore  linearly  separable,^ 
for  the  special  case  oi  P  =  2  tiro  first  bank  consists  of  a  nurnocr  of  TLU's 
connected  in  parallel.  Training  involves  determining  U  o  number  of  TLU's 
required  and  the  weights  associated  with  each  TI.U .  The  second  bank, 
consisting  oi  a  single  TI.U,  which  has  as  inputs  the  outputs  oi  the  first 
bank  ILU's,  is  trained  so  that  its  output  indicates  the  correct  response 
for  all  patterns  in  the  training  set. 
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A  linear  partition  of  pattern  space  is  formed  by  positioning  hyper¬ 
planes  in  the  space  so  that  the  space  is  divided  into  cells  such  that  no 
two  patterns  of  opposite  categorization  reside  in  the  same  cell.  A  non- 
rodundant  partition  is  defined  as  a  partition  with  the  property  that  if  any 
one  of  the  partitioning  hyperplanes  is  removed ,  at  least  two  non-empty 
cells  will  merg.  into  one  cell . 

4.2.1  Parallel  Hyperplane  Pirst  Stage  Implementation 

Given  P  hyperplanes  which  form  a  non-redundant  partition  of  two 
finite  subsets  of  patterns  vectors,  a  sufficient  condition  that  the  sets 
mapped  into  image  space  be  linearly  separable  is  that  exactly  P  +  1  cells 
formed  by  the  partition  be  occupied  by  patterns  . A  partition  of  pattern 
space  by  P  parallel  hyporplanes  fulfills  this  sufficiency  condition . . 

The  equation  of  a  hyperplane  may  be  written: 

C. -X  ~  z  (4-7) 

Given  the  hyperpiano  described  by  equation  (4-7)  in  E^  (C_  /  0) , 
it  may  be  staled  that  C.  is  a  vector  normal  to  the  hyperplane.  Any  vector 
leC  is  also  normal  to  the  hypcrplanc  (k  f  0)  .  The  two  vectors  of  unit 
length  C/ 1  C  |  ,  -C/I  C  |  are  the  unit  normals  to  the  hyperpiano. 

\Z-  |/|C  I  is  the  distance  of  the  hyperplane  from  the  origin.  Two  hyper¬ 
planes  are  parallel  if  they  have  the  same  unit  normal. 

We  may  form  a  non -i  euundcuil  puililiun  of  pattern  space  with 
parallel  hypeiplanos  by  the  implementation  of  the  following  algorithm: 
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1)  Select  a  unit  normal  vector,  N. 

2)  Form  the  dot  product  of  patterns  in  the  training  set  with  the 
unit  normal . 

3)  Arrange  the  dot  products  obtained  in  step  2)  in  order  of  value, 
largest  first . 

4)  Scan  the  ordered  dot.  products  to  find  adjacent  products  that 
belong  to  different  classes.  Lot  these  dot  products  (which 
are  distances  from  the  origin  along  the  unit  normal)  be 
designated  as  d(i)  and  d(i  +  1). 

5)  Place  a  partitioning  hyperplane  perpendicular  to  the  unit 
normal  at  a  distance  from  the  origin  in  the  direction  the  unit 
unit  normal  of: 

z.  =  [d(i)  +  d(i  +  1)3/2 

6)  Repeat  steps  4)  and  5)  until  all  the  adjacent  dot  products  of 
differing  class  have  been  found.  Let  P  be  the  total  number 
of  partitioning  hyperplanes  . 

In  terms  of  discriminant  iunction  wo  may  write: 

g,(X)  =  (-l)k+1  [N-X  -  z.]  (4-8) 

i  =  1 . P 

k  =  0  if  the  vector  associated  with 
d(l)  belongs  to  class  2 
k  -  1  if  the  vector  associated  with 


d(l)  belongs  to  class  1 


56 

It  is  seen  that  the  first  layer  of  our  machine  may  be  implemented  by 
P  TLU's.  Training  consists  of  finding  the  in  equation  (4-8)  and  is  non- 
p.obabilistic  in  form. 

Define: 

y.  =  1  if  g  (X)  >  0  (4-9) 

i  i 

-  -  ]  if  g.  (X)  <  0 

1  =  1,  .  .  .  ,  P 

The  vector  Y,  which  is  the  output  of  the  first  layer,  is  the  input 
to  the  single  TLU  which  comprises  the  second  layer  of  the  machine.  Since 
the  P  hyperplanes  have  formed  a  non-redundant  partition  of  pattern  space 
with  exactly  P  +  1  cells  occupied,  the  vectors  Y  in  the  training  set  are 
linearly  separable  and  the  discriminant  function  which  performs  the 
separation  may  be  written: 

g(Y)*W.Y+w  (4-10) 

g(Y)  >  0  =»  Xe  class  1 
g(Y)  <  0  =  X£  class  2 

It  can  be  shown  that  a  W'  which  satisfies  equation  (4-10)  is 
given  by: 

W  =  1  i  =  1 ,  2 . d 

Wd+1  =  0  P  odd  (4-11) 

,k+l 

=  (- 1 )  P  even 

where  k  is  as  defined  in  (4-8). 


57 


It  is  interesting  to  note  that  for  P  odd  the  discriminant  function  for 
the  second  layer  TLU  implements  a  majority  logic  decision. 

If  the  parallel  hyperplane  partition  approach  to  the  layered  machine 
is  used,  it  is  necessary  to  determine  an  efficient  means  for  choosing  a 
unit:  normal  vector.  It  would  be  desirable  to  position  the  unit  normal 
vector  so  that  the  projections  of  the  training  vectors  on  the  normal  vector 
are  close  together  for  patterns  belonging  to  the  same  class  and  far  apart 
for  patterns  belonging  to  different  classes. 

It  is  possible  to  obtain  such  a  vector  analytically  if  the  under¬ 
lying  probability  density  functions  are  known.  The  "discriminant  analysis" 
-technique  used  in  classical  statistics  does  this  for  vectors  from  popula¬ 
tions  with  normal  distributions  and  equal  variances  . 

Under  certain  assumptions  regarding  the  nature  of  the  data  (for 

example,  unimodality  of  the  underlying  probability  density  functions) 

1 3 

such  a  vector  may  be  estimated  from  experimentally  gathered  points 
However,  no  matter  what  method  is  used  to  arrive  at  the  unit  normal 
vector,  the  method  has  inherent  shortcomings.  One  may  visualize  any 
number  ol  cases  where  this  technique  would  fail  to  separate  even  dis¬ 
joint  classes  .  This  failure  may  be  traced  to  the  fact  that  multidimen¬ 
sional  data  is  effectively  being  reduced  to  a  sing’e  dimension  in  pattern 
space  with  an  accompanying  loss  of  information. 


Despite  the  obvious  drawbacks  the  algorithm  is  attractive  because 
of  its  simplicity  and  the  assurance  that  a  given  training  set  may  be 
separated  by  linear  surfaces.  Preliminary  preprocessing  of  the  data 
indicated  that  the  underlying  probability  density  functions  were  not  unimodal . 
No  cffoit  was  made.,  therefore,  to  obtain  a  unit  normal  vector  with  the 
desirable  properties  ennumerated  above.  The  algorithm  was  programmed, 
however,  using  the  difference  of  the  means  of  the  patterns  in  the  two 
classes  as  the  normal  vector.  The  results  are  ennumerated  in  Chapter  5 
of  this  report . 

4.2.2  Minimum  Distance  First  Stage  Implementation 

An  alternate  means  of  effecting  tho^requirod  partition  of  E^ 

(pattern  space)  takes  advantage  of  the  greater  "similarity"  of  patterns 
which  are  members  of  one  class  as  contrasted  to  patterns  which  are  mem¬ 
bers  of  the  other  class. 

Tor  the  purposes  of  this  dissertation  wo  shall  define  similarity 
in  terms  of  the  Euclidian  distance  between  points  . 

Define: 

d(X ,  P.)  =  [(X  -  P.)-  (X  -  P.)]^ 

~  ~i  1  i 

which  is  the  Euclidian  distance  between  X  and  P  .  If  d(X,  P  )  < 

_  —  j  —  — 1 

d(X,  P.)  we  shall  say  that  X  is  more  similar  to  P  than  to  P  . 

“I  -  -i  -j 

If  the  P.  are  prototype  points  as  defined  in  the  section  ol  the 

J. 

dissertation  dealing  with  the  minimum  distance  classifier  with  respect 
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to  point  sets,  a  similarity  decision  may  be  mads  using  the  discriminant 
functions  described  by  equation  (2—17) ,  repeated  below  for  convenience: 

g.(x)  -  X'F.  -  -P.  (2-17) 

i  l  i  i 

Since  (2-17)  set  equal  to  0  defines  a  hyperplane,  a  linear  partition  of 

E  has  been  formed.  There  is  nc  reason  to  expect  that  the  partition 

effected  is  non-redundant ,  however,  if  i  >  2  .  If  the  P  arc  colinear  the 

i 

partitioning  hyperplanes  are  parallel  and  the  situation  is  as  outlined  in 
Section  4.2.1.  If  the  P.  are  not  colinear,  more  than  a  single  dimension 
is  required  to  define  the  decision  regions.  We  may  logically  expect, 
.therefore,  that  w c  have  preserved  more  of  tho  information  contained  in 
t.he  original  measures  than  in  the  colinear  case. 

Suppose  that  our  situation  is  as  described  for  the  minimum 
distance  classifier  and  that  we  may  therefore  utilize  equation.  (2-19) 
as  a  discriminant  function  to  obtain  the  binary  representation  of  the  sub- 
segment.  pattern  required  as  the  output  of  the  pattern  classifier  first 
.stage.  Alternatively,  we  may  form  linear  discriminant  functions  for 
each  pair  of  P.  from  differing  classes  and  obtain  a  vector  h'ta  repre¬ 
sentation  of  the  pattern,  that  is,  effect  a  mapping  from  k ■  ,  urn  space 
to  a  binary  valued  image  space.  Such  a  mapping  may  be  accomplished 

by  a  bank  of  TLU's  .  If  nr  of  the  P  arc  prototype  points  from  class  1  and 

w. 

n  of  the  P  are  prototype  points  from  class  2,  a  totai  of  m  x  n  TLU's  will 
be  required.  A  binary  logic  network  with  the  first  bank  TLU  outputs  as 


N 

i 


inputs  may  be:  designed  which  will  implement  a  decision  identical  to  that 
obtained  by  the  use  of  the  discriminant  functions  described  by  equation 
(2-19).  In  either  case  the  decision  surfaces  in  pattern  space  aie 
piecewise  linear.  The  binary  logic  network  may  possibly  (but  not 
necessarily)  be  implemented  by  layered  TLU's. 

The  foregoing  discussion  has  presumed  the  existence  of  similarity 

or  clustering.  For  the  data  with  which  this  oapar  is  concerned, 

this  presumption  appears  to  be  valid.  Training  for  the  first  stage  of  the 

pattern  classifier  consists  of  the  determination  of  the  P  which  will  bo 

— i 

used  as  prototype  patterns  for  the  minimum  distance  classifier. 

We  have  defined  similarity  in  terms  of  closeness  of  points  in  the 
sense  of  Euclidian  distance.  A  reasonable  method  for  determination  of 
the  prototype  points  is  to  determine  the  centroid  of  points  which  show  the 
greatest  similarity.  This  requires  that  we  find  the  regions  in  pattern  space 
in  which  the  patterns  cluster. 

Two  b^sic  approaches  to  the  solution  of  this  problem  are  evident. 
One  approach  is  to  a  piioii  specify  the  boundaries  of  a  cluster  relative  to 
the  centroid.  Training  then  consists  of  locating  the  prototype  points  so 
that  all,  or  most,  of  the  training  patterns  are  contained  in  one  of  the 
clusters . 

A  second  approach  is  to  place  an  upper  bound  on  the  number  of 
clusters  which  will  be  considered.  Arbitrary  vectors  are  assigned  as  a 


first  approximation  to  the  centroids  of  the  assumed  clusters  and  training  is 
completed  by  adjusting  the  centroids  by  the  application  of  an  iterative 
algorithm . 

If  the  first  approach  is  chosen,  the  cluster  boundaries  will  define 
either  hyperspheres  or  hyperellipsoids  depending  upon  whether  the  assumed 
distance  of  the  boundaries  from  the  centroid  are  equal  in  all  directions  or 
are  ditferent  along  the  component  axes.  In  either  case  a  preliminary 
examination  of  the  data  will  be  necessary  to  determine  the  distances  to 
be  used, 

Algorithms  implementing  both  of  the  above  described  approaches 
were  programmed.  Descriptions  are  given  in  Chapter  5. 

4,2.3  Summary  of  First  Stage  Characteristics 

At  this  point  a  brief  summary  concerning  the  structure  of  lire  first 
stage  of  the  pattern  classifier  is  in  order. 

The  first  structure  considered  was  a  layered  machine,  the  first 
bank  of  which  consisted  of  TLU's  whose  hyperpiano  decision  surfaces 
wore  parallel.  It  was  shown  that  the  orientation  of  the  hyporplano  defined 
by  the  f’oconct  layer  TLU  Is  fixed  while  its  distance  from  the  origin  is 
dependant  only  upon  whether  the  number  of  first  bank  TLU's  is  odd  or  oven 
and,  if  the  number  of  the  first  bank  TLU's  is  oven,  on  the  class  member¬ 
ship  of  the  training  pattern  which  is  iuitherc.si  from  the  origin, 
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Binary  representations  of  the  pattern,  X,  arc  available  as  the  output 
of  the  first  layer  TLU's  and  as  the  output  of  the  second  layer  TLU's.  It  was 
noted  that  a  given  set  of  patterns  could  be  separated  by  this  structure,  but 
that  second  layer  decision  errors  would  be  expected  for  patterns  not  in¬ 
cluded  in  the  training  set. 

The  second  structure  considered  was  a  minimum  distance  classifier 
with  respect  to  point  sets.  Training  for  this  s'ructure  consisted  oi 
estimating  tire  prototype  patterns.  It  was  noted  that  two  implementations 
of  the  minimum  distance  classifier  were  possible.  The  first  utilized  the 
discriminant  function  given  as  equation  (2-19)  while  the  second  consisted 
of  a  first  bank  of  TLU's  feeding  a  binary  switching  circuit.  The  decisions 
reached  by  the  first  stage  were  identical  for  the  two  structures. 

r or  both  the  minimum  distance  and  parallel  hypeiplane  classifiers, 
then,  two  binary  representations  of  a  pattern  subsegment  are  available. 

The  first  is  a  binaiy  valued  vector  which  has  as  elements  the  output  of 
a  first  bank  of  TLU's.  The  second  representation  is  a  single  binary  value 
representing  the  fust  stage  uOuL-Uou. 

These  binary  representations  are  the  input  to  the  probabilistic 
second  stage  of  the  pattern  classifier. 

4.3  Pattern  Classifier  Probabilistic  Par, end  Stage 
4.3.1  Statistically  Independent  Measures  Model 

To  this  point  wc  have  been  concerned  with  the  ordered  set  of  measures 


awwfmm . . . — . . . . . . . . 
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obtained  from  a  single  1024  sample  subsegment  of  the  original  digitized 
data.  Define  as  a  first  stage  binary  representation  of  the  jth  subseg¬ 
ment.  Suppose  a  segment  contains  k  subsegments.  Define: 

(k) 


z  =  (x^  , 


,  X  ) 


Utilizing  the  logarithmic  form  of  the  Bayes  discriminant  function 

( p 

for  the  symmetric  loss  function  with  R  =  2  and  assuming  the  X  statisti¬ 
cally  indpendent,  we  may  write: 

g(Z)  =  i  In  +  -On 

1  =  1  p(X  ''  1 2)  P(2) 

If  the  elements  of  are  assumed  to  be  statistically  independent 
the  following  relation  applies: 


k  d  I  ^  p(j ) 

g(Z)  =  X  X  In  - — — -  4  In  L1LL 


1  p(x 


0) 


?.) 


p(2) 


(4-12) 


Define  the  subsidiary  discriminant  functions: 


(J) 


(X)  =.E  In 

y-l 


p(x.(j)  |  1) 
P  (x .  |  2) 


(4-13) 


liquation  (4-13)  is  of  the  form  of  equation  (4-1)  with  p(i)  -  p(2) 
(maximum  likelihood  discriminant)  . 

Substituting  (4-13)  into  (4-12)  yields: 


y(Z)  =  X  y(j)  (X)  4  In 

J-l  P  (2) 


(4-14) 


Extending  the  nomenclature  previously  used  to  define  the  various 


probabilities,  we  define: 


P(x.<j)  =  1  1 1)  -  p.(j) 
p(x.(j)=  0  I  1)  -1  -  P,(j) 
p(x,0)  =  1  j  2)  =  q,0' 
p(x.'^  =  0  I  2)  =  1  -  q.(j) 

l  i 


Then: 


d 

(X)  =  X  x.  f-n 
r=l  1 


Oh 


O'), 


d 

+  X  -On 
i=l 


U 

0 


0) 

ol 


) 

) 


Substitution  of  (4-15)  into  (4-14)  yields: 


k  d 

g(Z)  =  £  Ex,  -On 

“  j=l  i=l  3 


o) 


n 


0), 


k  d 

4  X  X  •On 
1=1  1=1 


0 

<1 


-P.0)) 


H  -til 


dU 

p(2) 


d  s  dimension  of  X 
k  =  No,  subsegments 


(4-1 


(4-16 


Equation  (4-10)  is  of  the  foim: 
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g(z)  —  w. z  +  w . 


where 


'6+1 

W  and  Z  6  dimensional 
6  =  d  x  k 


P^O  “ 


(4-17) 


w  =  w  =  6n 
m  j  xd+ i 


n0,d  -P,(i)) 


j  =  1 . k 

i  =  1 ,  ...  ,  d 
m  =  1 ,  2 ,  ...  ,  l 


(4-18) 


6  -  d  x  k  ... 
k  d  (i  -p.Uj) 

w  =  z  z  - — - 

il'i  J-l  1=1  ()• 


+  <•„  ^ 


Training  then  consists  of  estimating  the  p, 


1  -  P(i) 
(j)  .  (J) 


q. and  p(l) , 


(i  -  l . cJ  ;  j  =  1 . k  ) . 

4.3.2  Markoff  1  Distributed  Measures 

The  presumption  of  statistical  indpendence  invoked  in  Section 
4,3.1  was  justified  on  tiia  basis  of  computational  manor, -ability .  If  the 
elements  of  the  binary  valued  vector  represent  time  sequential  first  stage 
decisions,  a  possibly  better  assumption  is  that  the  measures  comprise  a 
Markoff  chain . 

The  general  logarithmic  form  of  the  two  class  symmetric  loss  Bayes 
discrumriuiit  function  is  repeated  below: 


66 


If  (x^ ,  x  } 


g (X)  =  -6n  p(x|  1)  -  In  p (X 1 2 )  + 


P  (2) 


P(X  |  j)  =  p(xt,  x2 . xk  |  j) 

j  =  1,  2 


(4-19) 

(4-20) 


k  -  number  of  sequential  first  stage  decisions 

11 


x  )  is  a  Markoff  chain,  by  definition  : 
k 


p(x  Xj_2 '  '  x<)  =  p(*j  Ix^j) 


(4-21) 


i  =  2,  3,  .  .  .  ,  k 

Then,  by  the  definition  of  conditional  probability: 


P(Y  \-i 


V  °plxk  ixK-ilp(xk-rxk-2 . V 


“  P!xk|xk-l)  P(xk-1  1  Xk-2'  P<\-2 . Xl> 


P<V  xk-l . *,>'p<*,>,4*  p(xJx,-l)  «**» 

Substitution  of  (4-22)  into  equation  (4-19)  yields: 


p(Xj  1 1)  k  p(x 1 1 ,  ) 

[T(x  ,  1 2  ,  x*  , ) 


9(X)  =  In  ~7 T— FT  ln 


+  -tn 


1 

iilil 


i  =  2 


1  -  P(D 

Recall  that  X  may  take  on  values  of  only  1  and  0  ,  Define; 


P(x1=  1  1 

1)  =  p 

p(x.  —  1  i 

1  , 

1  Vl  ■ l'  - 

1 

p(x.  =  1 

1  v  A 

1  ,  x.  =  0)  = 

1  i-l 

(4-23) 


I 


. . . 


p(x.  =  0 

1  ^  Vi 

p(x.  -  0 

1  1.  Vl 

P(x  =  1  | 

2)  =q 

> — 1 

11 

>c 

a 

l2'  Vl  ' 

p(x.  =  1 

1  2'  Vl 

p(x.  =  0 

1  2,  x.  , 
l-l 

P(x.  =  0 

,2' Vi 

i  = 

A 


A 

—  \ 

A 


0)  =  1  -  v. 


i=2,. 


Making  use  of  the  fact  thatx^  -Oor  x.-l  we  may  write: 


£n 


P(x,  I  1) 
_ 1. 

P(x'  \2) 


x.  [  X.  ,  { n  - 

i  L  t-1  t. 


+  [1— X..I  [x._1  In 
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(4-24) 


(l  '  Vi)  ^  T~  i 


1 -r.  1-s. 

r=r.  +u  -Vi)tn  r^r 


S .  (1  -v.) 

-  x.  -611  — dv - % 

l  v .  (1  -s  .) 

1  1 

r.v.  (1  -t.)  ( 1  -s  .) 

.  1  J _ 1 _ 1 

x .  x .  / ,  \  / 1  \ 

l  i-l  t.s.(l-r.)(J  -v.) 

li  i  i 

(l-r.)d-v.) 

+  Xi-1  ln  (]-t.)(l-s.) 


(4-25) 


+  -In 


1-s. 


1  —  v . 


6P 


Also: 


I  ^  p  ()  -p) 

■f-n  T^TTi  =  *,  ln  L  4  0“X.)  Hn 

p(x2  |  2)  J  q  1  (1-q) 


(4-26) 


=  Xl  tn  -ellisJL  +  .ILzeL 

Xltn  q(]-p)  +  *“  (1-q) 


Substitution  of  equations  (4-25)  and  (4-26)  into  equation  (4-2 


3) 


yields: 


P(l-q)  k  iy  V 

g(X)  =  x,  In  4  >-  x-  -J-77— 

W  1  q(l-p)  i=2  1  v.(l-s 

L-  r.v  (1  -t.)(i  -s.) 

,  V  „  v  1  1  1 _ i_ 

i=2 '  1  i-1  *  t,s.(l  -r.)(l-v.) 

^  ill  l 

(l-r)(l-v.) 


+  £  x  ,  tn  ..  ,  w, - ; 

i=2  1-1  (l-t)(l-s) 


k 

+  r.  In 

1=2 


1  -s . 

_ i_ 

1  —v . 


(4-27) 


+  -tn 


ldB_ 


+  -tn 


1-q  '  ■“  l-p(l) 

If  k  is  fixed,  equation  (4-27)  may  be  witten  as: 

p(l-q)(l-r  ) (1  -v  ) 
y(x)  =  x,  -tn  rTT-rirr— rrr~ r- 

i  _l2mj  21 


k-1 

+  E  x. 
i=2  1 


s.(J  -v.)  (1-r  )(1  -v  ) 

1  1 _ H  J _ 14  1 

v  (1  -s  .)  (1  -t  )  ( I  -s  ) 

1  1  14  1  1 1-  ] 


+  X.  Hi 


S,  <1-V,  ) 


k  Vk(l-S|[) 


(liquation  cont'd  on  next  page) 


I  iiMWIi  h  UiiiiiiiMW  JiliiiHlIHIHl  Wiiwi  ,ilMI,iiliiHiiwm#|i,t||ii'i  i 


1 


j,  r.  v.  <  1  — t .)  ( 1  — s .) 

<-o  i  l-l  t.  s.  ( 1  — r , ) ( 1 —v , ) 
1-4  ii  l  l 


+  E  In  ~ - V-  +  -tn  p +  -Ln  P^l\,s  (4-28) 

(1-v.)  1-q  l-p(l) 

i  jL  i 


Equation  (4-28)  is  of  the  form: 


y  (X)  —  \V  .  X  l  A  .  i  (X)  4-  v 


where; 


Cp.  ,  (X)  =  x.  * 


J  J-l 


J  =  2 ,  .  .  . ,  k 


(4-30) 


cpk  (x)  -  0 


Since  x  is  binary  valued: 


x.  x.  =  0  for  x.  /  x.  ,  /  1 


j  j-l 


j  J-l 


(4-31) 


x,  x.  =1  for  x.  =  x.  =  1 


j  j-l 


j  j-l 


Equation  (4-31)may  be  implemented  by  a  TLU;  equation  (4-29)  may 


therefore  be  implemented  by  a  layered  machine.  Since  a  layered  machine 


implements  a  piecewise  linear  discriminant  function,  g(X)  in  equation 


(A  O  n)  \  n  ^  n  1  T  f  il  in-iiiAv\  S  «—  II  nnf)  =»r>  tKn  r«  r*  O  VW  ] 

V  t  Zi  -J  /  a  vj  v.-  v  .■  v  v  a  or;  uuoui  .  u  uuo  i  vuli^u  u.Uii  i.  o  u  j  «.»Jw 


stage  of  a  pattern  classifier  for  which  the  first  stage  is  piecewise  linear 


(for  example  the  first  sia^e  described  in  Section  4.2  of  this  paper)  . 


the  composite  machine  is  piecewise  linear. 


Training  for  the  Markoff  machine  consists  of  estimating  the 


U 


quantities  defined  in  (4-24). 


. iiuij  HjULiiii'LJiui- 


Define: 


P  =  Number  of  class  1  training  patterns  for  which 


Rj  -  Number  of  class  1  training  patterns  for  which 

x  ,=  1  and  x  ,  =  1 ;  i  2,3,  .  .  .  ,  m . 
i  i  -] 

-  Number  of  class  1  training  patterns  for  which 

x  =  1  and  x  ,  =  0;  i  =  2  ,  3 ,  .  .  .  ,  m . 
i  i-1 

Q  --  Number  of  class  2  training  patterns  for  which 


X]  =  1  ' 

T.  =  Number  of  class  2  training  patterns  for  which 
x.  '=  1  and  x  =  1;  i  =  2,  3,  ...  ,  m. 

V  =  Number  of  class  2  training  patterns  for  which 
x.  =  1  and  =  0;  i  •=  2  ,  3 ,  .  .  .  ,  m. 

N^  =  Number  of  class  1  training  patterns  having 
at  least  j  components  (assuming  that  the 
patterns  can  have  a  varying  number  of  com- 


ponents);  j  “  ! ,  2 , 


/  A 


N  .  =  Number  of  class  2  training  patterns  having 

£  J 

at  least  j  components;  j  =  1,  2,  .  .  .  ,  m. 


M  ^  =  Number  of  class  1  training  patterns  for  which 
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=  Number  of  class  2  training  patterns  for  which 
x  =  1  ,  i  =  2 ,  3  ,  .  .  .  ,  m . 

ni  =  Minimum  of  the  maximum  number  of  components  in 
class  1  training  patterns  or  class  2  tiaining 
patterns  . 

Reasonable  estimates  for  the  quantities  defined  in  (4-24)  are: 


P  = 


P/N 


11 


r.  =  R/M,. 

l  l  li 


s  .  =  S  /(N  -M  . ) 
i  i  1 1  1 1 


q  =  Q/Ngi 


t.  =  T./M 

1  1 


21 


(4  "33) 

v.  =  V./(N  -M  ) 

1  i  2i  2i 


i  =  2  ,  3 ,  .  .  .  ,  m 

If  k,  the  number  of  components  in  a  pattern,  is  fixed,  equation 
(4-28)  may  be  used  as  the  recognition  discriminant  function.  If  k  is 
variable  from  pattern  to  pattern  it  is  necessary  to  use  equation  (4-27)  to 
effect  recognition  since  it  is  not  possible  to  compute  the  constant  term 
until  the  dimensionality  of  the  pattern  is  known. 

4 . 4  Compos  its  Machine  Structure 

Recall  that  the  first  stage  of  the  pattern  classifier  described  in 
Section  4.2  maps  the  pattern  into  a  binary  representation  .  These  binary 
values  may  be  either  0  or  1  or+1  and  -1  ,  depending  upon  the  representa¬ 
tion  required  for  the  succeeding  stage. 

Figure  4-1  is  a  block  diagram  of  tire  pattern  classifiers  implemented. 
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Referring  to  this  figure,  is  a  vector  consisting  of  d  real  valued  measures 
for  the  ith  subsegment  in  a  pattern.  There  are  a  total  of  k  subsegments  in  a 
pattern.  Z  is  therefore  a  c:xk  dimensional  vector  which  repiesents  a  complete 
pattern . 

Assume  that  the  first  stage  of  the  first  layer  of  the  pattern  classi¬ 
fier  consists  of  p  TI.TT's,  is  a  binary  valued  p  dimensional  vector 

representing  the  ith  subsegment  of  the  pattern.  V  is  the  first  stage-- 
first  layer  binary  representation  of  a  total  pattern  and  consists  of  pxk 
ordered  binary  values. 

The  output  of  the  second  layer  of  ihe  first  stage  of  the  patLern 
.classifier  is  a  binary  valued  scalar  u.  which  represents  tho  pro- 
liminaiy  decision  made  by  the  first  stage  of  tho  pattern  classifier  as 
to  the  classification  of  the  ith  subsegmen)  of  a  patten..  U  is  a  k 
dimensional  binary  valued  vector  which  repiesents  the  preliminary 
sequential  classification  decision.;  of  the  first  stage  of  the  pattern 
classifier  for  a  complete  pattern. 

Regardless  of  tho  form  that  the  pattern  classifier  takes,  errors 
in  classification  arc  to  bo  expected.  It  is  logical  to  c  ssume  that  an 
Increase  in  the  complexity  ot  tho  machine  chosen  would  be  reflected  in 
a  lowering  of  the  piobabili’  of  erroneous  classification. 

Tive  second  stage  classifiers  were  implemented.  The  differences 
in  machine  complexity  will  be  apparent  from  tho  desciiptions  of  the 
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machines.  The  structures  of  the  machines  follow  from  tha  presumption 
made  concerning  the  statistical  properties  of  the  first  stage  binary  vectors. 
These  presumptions  arc  outlined  rn  the  sections  of  this  chapter  of  the 
dissertation  which  follow. 

4.4.1  Machine  Number  1 

Referring  to  riguie  4-1,  machine  number  1  operated  on  U,  the 
first  stage — second  layer  output.  This  implementation  was  made  under 
the  assumption  that  the  u.'s  were  statistically  independent  and  identically 
distubutcd  for  i  =1,  2,  .  .  .  ,  k.  The  applicable  discriminant  function 
is  given  below  as  equation  (4-34): 


g  (u)  =  >:  u  tn  +  k  t„  fai  H-  fin—. 

1  i=]  ]  q(]-p)  1-q  j  -P(J)  (4_34) 

p  and  q  were  estimated  from  a  tiaininy  subset  without  regard  to  subseg¬ 
ment  number .  It  Is  seen  that  only  three  values  must  be  stored  for  the 
second  stage  classifier. 

4.4.2  Machine  Number  2 

Referring  to  Figure:  4-1  ,  machine  number  2  operated  on  V,  the 
first  stage --first  bank  binary  representation.  This  implementation  was 
made  under  the  presumption  that  the  y  ^'s  were  statistically  independent 


over  j  -  l,  2,  .  .  .  ,  p  and  identically  distributed  for  i  =  1,  2,  .  ,  .  ,  k.  The 
applicable  discriminant  function  is  given  below  as  equation  (4-35) : 
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k  p  iL\ 

g„(V)  -  T,  T.  y,'1'  in 
2  i-1  j-1  J 


pi°~qi) 


P 

+  k  r.  in 

j  =  l 


(1-p.) 

(1-Q.) 


•+  in 


Eli) _ 

l  -PCD 


(4-35) 


The  p.  and  q  were  estimated  from  a  training  subset  without  regard  to  sub- 
J  J 

segment  number . 

4.4.3  Machine  Number  3 

Referring  to  figure  4-1.  machine  number  3  operated  on  U,  the 
first  stage-second  layer  output.  This  implementation  was  made  under 
the  assumption  that  the  u.'s  were  statistically  independent,  but  not 
necessarily  identically  distributed  tor  i  =  1 ,  2  ,  .  .  .  ,  k.  The  applicable 
discriminant  function  is  given  below  as  equation  (4-36): 


k 

g  (U)  =  bu  -t.il 
a  1  =  1  1 


P.(l  ~q  ) 


k 

-I  E  -tn 
i=i 


d-P,,) 

(1  -q;) 


-t  -tn 


illil _ 

1-P(1) 


(4-36) 


The  p  and  q  were  estimated  from  a  training  subset  using  only  the  ith 
subsegments  to  estimate  p^  and  q^. 

4.4.4  Machine  Number  4 

Referring  to  I’igure  4-1,  machine  number  4  opeiated  on  V,  the 
first  stage--fiisl  bank  binary  represontation .  This  implementation  was 
made  under  the  pro  sumption  that  the  y/^'s  wore  statistically  independent 


i 

i 


I 
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for  i  =  1,  2,  .  .  .  ,  k;  j  =  l,  2,  .  .  .  ,  p,  but  not  necessarily  identically 
distributed  over  i  .  The  applicable  discriminant  function  is  given  below 
as  equation  (4-37): 


g4(v) 


E  fv  (l)  in 

jC 

j=iL  J 

(i.p.w)  1 

_ j _ 

■  p 

i 

v-i. 

1  _ 

+  Vfl 

(4-37) 


l-p(l) 


p/  ^  and  q  ^  ^  were  estimated  using  only  jth  first  bank  TLU  output  for  the 
ith  subsegment  in  the  training  subset. 

4.4.5  Machine  Number  5 

Machine  number  5  is  identical  to  machine  number  3  except  it  was 
assumed  that  the  u^'s  were  Markoff  1  distributed  rather  than  being  statisti¬ 
cally  independent.  the  applicable  discriminant  function  is  given  below  os 
equation  ( 4-313) : 

p()  -q)(]  -r  ) (1  -v  ) 

gs  od  -  u2  in  q()_p)(]:t^a_£^- 

,  si(J-Vi)(l-ri+l)(l_Vl+]) 

'^Ui  11  v.a-^a-t^ ,)(]-«) 


sk  (1-vk) 

+  U  -In  - — - T— 

k  vk  (l  -sfc) 

(equation  continued  on  next  page) 


(4-38) 
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i=2  1  1_1 


r.v.(l  -t .) ( 1  -s.) 
i  i  i  i 


t.s.O  -r.)  (1  -v.) 
li  l  l 
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+  T.  -tn 
i-2 


(1  -s.) 
(1-v.) 


+  -tn 


1  -p 

—  4  -tn 
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i-p(i) 


The  various  probabilities  were  estimated  by  use  oi  the  relationships 
given  in  (4-32)  and  (4-33)  with  respect  to  u. 
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CHAPTER  V 


RESULTS 

5.1  Data  Processed 

As  outLined  in  section  3 . 2  of  this  report,  data  processed  was 
from  one  of  two  categories.  These  categories  were: 

Category  one  —  data  recorded  aL  Woodtawn  Hospital 

Category  two  --  simulated  coughs  and  artifacts  recorded  at 
The  University  of  Texas  at  Austin. 

Category  one  data  was  taken  from  recordings  made  without  benefit 
of  the  start-stop  recording  procedure  outlined  in  section  3.2.1  .  Rather, 
the  original  audio  tape  recording  was  edited  and  re-recorded  prior  to 
digitization.  Included  in  this  type  of  data  wore  forty -one  cough  segments 
(patterns)  which  contained  a  total  of  444  sub-segments  and  twenty-eight 
artifact  segments  which  contained  a  total  of  342  sub-segments  (a  total  of 
sixty-nine  segments  and  786  sub-segments). 

Category  two  data  included  112  cough  segments,  which  contained 
450  sub-segments  and  twenty-eight  artifact  segments  which  contained 
850  sub-segments,  •  total  of  140  segments  and  1  350  sub-segments. 

When  category  one  data  was  used  for  training,  it  was  possible  to 
estimate  the  a  priori  probability  of  occurrence  of  the  cough  and  artifact 
classes.  In  the  case  of  category  two  data,  such  an  estimation  was  not 
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possible,  since  the  true  relative  number  of  occurrences  of  cough  and  j  j 

:  I  1 

artifact  patterns  was  not  preserved.  The  a  priori  probabilities  for  this 

latter  case  were  set  equal,  which  resulted  in  maximum  likelihood  decisions  ; 

rather  than  Bayesian  decisions.  |  I 

The  discrepancy  between  the  lengths  of  the  category  one  and 
category  two  data  signals  was  due  to  the  recording  techniques  employed. 

The  background  noise  in  category  one  data  tended  to  make  the  coughs  j 

longer  since  the  individual  "hacks"  which  constituted  a  cough  series  {  ! 

were  placed  in  the  same  segment  in  many  cases.  The  category  two  arti- 

i  : 

fact  segments  tend  to  contain  many  sub-segments  because  they  were  j 

recorded  from  continuous  commerical  recording  signals. 

The  differing  lengths  of  signal  did  not  have  a  significant  effect 
cn  training  and  classification  results,  however,  since  it  was  required 
that  sufficient  numbers  of  sub-segments  be  available  from  both  the  cough 
and  artifact  classes  to  obtain  reasonable  estimates  oi  the  probabilities 
involved.  During  the  classification  phase,  sub-segments  in  excess  of 
this  number  were  ignored. 

5.2  Measure  Calculation 

Measures  were  calculated  by  the  procedures  outlined  in  section  3.3 
of  this  paper. 

Computation  of  the  measures  for  a  102-1  point  sub-segment  required 
approximately  0.8  second,  of  which  approximately  0.7  second  was  expended  i 

on  calculation  of  the  1’ourier  seiies  coefficients.  The  measure  numbers  to 

I 

li 
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which  references  are  made  in  the  discussion  which  follows  are  listed  in 
Tabic  5-3. 

Prior  to  utilization  of  measure  number  139  (average  zero-crossing 
frequency)  and  measures  140  through  228  (spectral  analysis  measures), 
it  was  necessary  to  normalize  the  calculated  values  so  that  tire  resultant 
measures  had  a  nominal  maximum  value  of  one.  Failure  to  effect  this 
normalization  would  tend  to  place  emphasis  on  those  measures  with  the 
largest  magnitude. 

The  empirical  normalizing  relationships  used  were  as  follows 
(v  is  the  calculated  value  of  the  measure): 

Measure  No.  139  (average  zero-crossing  frequency)  --(Iog^v)/I6500 

Measure  Nos.  140-1  48  (magnitude  squared  Fcumer  coefficient 

-3 

bands)  -  8.33  x  10  log^v 

Measure  Nos.  149-1  68  (frequency  at  which  spectral  peaks  occurred) 

and  measure  Nos.  1  09-220  (spectral  peak  1/4  power  frequencies) 
0.025  log  v 

Measure  Nos.  169-1  88  (magnitude  squared  of  spectral  peaks)  - 
0 . 01  log^v  . 

5.3  Mousuie  Selection  Results 

T.h e  procedure  outlined  in  section  3 .4  wci s  applied  to  category  one 
data  and  category  two  data  separately.  Table  5-1  is  a  list  of  the  results 
obtained  foi  the  fifty  highest  tanked  measures  for  category  one  data  and 
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TabLe  5-2  for  category  two  data.  The  criterion  value  in  columns  three  and 

CO 

six  of  the  tables  is  an  estimate  of  f  |p{v/l)  -p(v/2)  |dv  where  p(v/l)  and 

—CO 

p(v/2)  are  the  inarignal  probability  functions  conditioned  on  the  measure 
v  having  originated  from  a  signal  belonging  to  the  cough  and  artifact 
classes  respectively.  The  measure  numbers  in  columns  two  and  four  are 
identified  in  Table  3-5.  IL  will  be  noted  that  a  maximum  possible  criterion 
value  of  2.0  would  indicate  that  the  classes  were  disjoint  with  respect  to 
that  particular  measure. 

The  measures  listed  in  Table  5-1  will  be  referred  to  as  "measure 
set  A,"  The  measures  listed  in  Table  5-2  will  be  referred  to  as  "measure 
set  C."  The  first  twenty -five  of  the  measures  listed  in  Table  5-2  will  be 
referred  to  as  "measure  set  D." 

Because  category  one  data  contained  considerable  noise,  it  would 
be  expected  that  the  spectral  features  would  be  masked  to  some  extent. 
This  conclusion  is  verified  by  the  feature  selection  results.  For  this  group 
of  data,  the  features  emphasized  are  the  envelope  difference  and  zero- 


-*  VO  <-(-!>■ 


ICO  t  C41  I 


w hiio  the  spectral  features  Uiu  aliuOSL  tiu in ^ i y  excuiuou  . 


In  the  case  of  category  two  data.,  for  which  the  signal  was  relatively 
clean,  spectral  features  predominate  as  those  most  likely  to  aid  in  recogni¬ 
tion.  The  amplitude  density  and  envelope  amplitude  density  fealurcs  are 
also  stressed. 

The  estimated  absolute  difleiunce  in  marginal  densities  indicates 
that  recognition  error  rates  should  be  considerably  lower  than  those  for 


82 


Table  5-1 

Measure  Selection  Results 
Category  One  Data 


Rank 

Feature 

Number 

Criterion 

Value 

Rank 

Feature 

Number 

Criterion 

Value 

1 

209 

0.727 

1  8 

191 

0.377 

2 

149 

0.721 

19 

12 

0.377 

3 

1  89 

0.701 

20 

151 

0.374 

4 

138 

0.672 

21 

94 

0 .364 

5 

48 

0.640 

22 

137 

0 . 363 

6 

135 

0.  G2G 

23 

102 

0.362 

7 

136 

0.622 

24 

106 

0.3  60 

8 

97 

0.4CG 

25 

40 

0.359 

9 

92 

0.4G6 

26 

87 

0.356 

10 

144 

0.451 

27 

9  3 

0.348 

11 

47 

0.447 

28 

105 

0.345 

12 

49 

0.414 

29 

18 

0.333 

13 

4  6 

0,392 

30 

218 

0.333 

14 

90 

0 . 3  87 

31 

104 

0.332 

15 

211 

0,380 

32 

17 

0 .329 

1G 

103 

0.304 

33 

1  50 

0.327 

17 

90 

0.3  84 

34 

107 

0.324 
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Feature 


Rank 

Number 

35 

198 

36 

10 

37 

1  39 

38 

100 

39 

05 

40 

50 

41 

44 

42 

91 

43 

3.1 

44 

99 

45 

51 

46 

88 

47 

24 

48 

95 

49 

108 

50 

45 

Criterion 

Value 

0.321 

0.317 

0.314 

0.314 

0.310 

0.310 

0.305 

0.301 

0.300 

0 . 299 

0.296 

0.291 

0.289 

0.283 

0.2  83 

0.282 
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Table  5-2 

Measure  Selection  Results 
Category  Two  Data 


Rank 

Feature 

Number 

Criterion 

Value 

Rank 

Feature 

Number 

Criterion 

Value 

1 

11 

1 .124 

18 

176 

0.920 

2 

12 

1.105 

19 

173 

0.913 

3 

17 

1  .099 

20 

20 

0.913 

4 

16 

1  .070 

21 

48 

0.911 

5 

21 

1.068 

22 

175 

0.911 

6 

22 

1.048 

23 

174 

0.900 

7 

10 

1  .007 

24 

179 

0.903 

8 

49 

0.904 

25 

23 

0.897 

9 

148 

0.942 

26 

170 

0.896 

10 

177 

0.940 

27 

186 

0.892 

1 1 

1  3 

0.938 

28 

146 

0 . 889 

12 

170 

0.938 

29 

1  85 

0.889 

13 

171 

0.936 

30 

1  84 

0.887 

14 

147 

0.933 

31 

1  80 

0.883 

15 

142 

0.927 

32 

181 

0.879 

16 

169 

0.927 

33 

187 

0.87  7 

1  7 

172 

0.925 

34 

183 

0,877 
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Rank 

Feature 

Number 

Ciuteiion 

Value 

35 

188 

0.868 

36 

182. 

0.860 

37 

9 

0.849 

38 

144 

0.849 

38 

189 

0.836 

4  0 

24 

0.825 

41 

145 

0.802 

42 

47 

0.800 

43 

8 

0.791 

44 

54 

0.791 

45 

25 

0,726 

46 

50 

0.721 

47 

41 

0.709 

48 

43 

0.702 

49 

42 

0.684 

50 

7 

0. 645 

36 


category  one  data.  Results  presented  in  section  5.5  show  that  such  is 
indeed  the  case. 

An  additional  set  of  measures  was  selected  on  the  basis  of  simplicity 
of  implementation  of  the  pattern  classifier  by  special  purpose  circuitry. 
Zero-crossing  measures  and  envelope  difference  measures  were  promising 
candidates  in  the  light  of  the  measure  selection  results  for  category  one 
data.  The  twenty-eight  measures  chosen  were  measures  number  85  through 
198  and  measure  numbers  135  through  138.  These  measures  will  be  referred 
to  as  "measure  set  B." 

5.4  Training  Set  Selection 

The  first  stage  of  tire  pattern  classifier  was  trained  without  regard 
to  the  location  of  a  sub-segment  within  a  segment.  The  second  stage  of 
the  classifrei  ,  on  the  other  hand,  took  the  order  in  which  the  sub-segments 
occur  into  account.  Two  types  of  training  classes  were  therefore  required; 
the  first  consisted  of  unorderod  sub-segments  and  the  second  of  complete 
segments  (patterns). 

Training  and  classification  were  made  for  each  of  the  two  categories 
of  data  separately.  A  set  of  training  patterns  was  therefore  chosen  from 
each  ol  the  categories. 

The  procedure  used  for  selecting  the  training  sets  is  given  below: 

1)  A  specilied  percentage  ol  the  patterns  (segments)  belonging 
to  each  of  the  classes  (artifact  and  couyh)  were  selected  at 
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random.  Those  constituted  th  training  set  and  were  used  to 
train  the  second  stage  of  the  pattern  classifier. 

2)  A  specified  percentage  of  the  sub-segments  contained  in  each 
of  tiie  patterns  selected  in  l)  were  chosen  at  random  to  form 
a  training  sub-set.  This  sub-set  of  training  sub-segments 
was  used  for  training  the  first  stage  of  the  pattern  classifier. 

The  relative  frequency  of  occurrence  of  coughs  and  artifacts  is 
preserved  by  the  above  training  set  selection.  Use  of  the  training  set  to 
estimate  a  priori  probability  of  occurrence  is  therefore  valid  (for  category 
one  data) . 

Seventy-five  per  cent  of  the  available  category  one  patterns  were 
selected  for  the  training  set.  This  set  consisted  of  thirty-one  cough 
patterns  and  twenty-one  artifact  patterns,  a  total  of  fifty-two  patterns 
(out  of  an  available  sixty-nme). 

Fifty  per  cent  of  the  sub-segments  from  each  of  the  training  patterns 
were  selected  for  the  training  sub-set.  Tire  sub-set  consisted  of  164 
cough  sub-segments  and  132  artifact  sub-segments,  a  total  of  29  G  sub¬ 
segments  . 

The  category  two  data  training  set  consisted  oi  Jilty  per  cent  of 
the  available  patterns.  The  set  contained  fifty-five  cough  patterns  and 
four  teen  artifact  patterns,  a  total  of  sixty -nine  patterns. 

The  training  sub-set  fur  category  two  data  was  made  up  oi  50% 
of  the  sub-segments  from  each  of  the  training  patterns  -  10  8  cough 
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sub-segments  and  20G  artifact  sub-segments,  a  total  of  314  sub-segments. 
5.5  Machine  Training  and  Classification  Results 

Training  and  recognition  using  categoiy  one  data  were  accomplished 
for  four  first  stage  configurations  used  in  conjunction  with  four  second  stage 
configurations;  the  Markoff-1  machine  was  not  implemented.  Training  and 
recognition  using  category  two  data  were  conducted  for  the  first  stage  con¬ 
figuration  chosen  as  a  final  first  stage  and  for  all  five  second  stage 
configurations.  The  second  stage  implementations  were  as  described  in 
sections  4.4.1  through  4 . 4 . 5 . 

The  first  stage  configurations  utilized  are  listed  below: 

1)  Parallel  hyperplane  implementation  (as  described  in 
section  4.2.1). 

2)  Modified  parallel  hyperplane  implementation 

3)  Minimum  distance  with  respect  to  point  sets  configuration 
with  pre-sot  cluster  boundaries  (described  in  section  4.2.2) 

4)  Minimum  distance  with  respect  to  point  sets  configuration 
with  boundaries  determined  by  an  iterative  alyoiithin  (described 
in  section  4.2.2). 

The  first  stage  configuration  listed  as  4)  above  was  chosen  as 
a  final  tirst  stage  for  the  pattern  classifier. 

Training  and  recognition  lesulls  aie  tabulated  in  Tables  5-3 
thiough  5-14.  The  following  abbreviations  were  used; 
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A  - Artifact  Class 

C - Cough  Class 

D/D  --  a  decision  was  made  that  the  pattern  belonged  to 

class  B  when  in  actuality  it  was  a  member  of  class  D. 

B  =  A,  C  ;  D  =  A,  C 

The  data  identification  and  measure  set  nomenclature  previously 
defined  are  repeated  below  for  convenience: 

Category  one  data - recorded  at  Woodlawn  Hospital 

Category  two  data - simulated  coughs  and  artifacts 

Measure  set  A  - the  fifty  measures  described  in  Table  5-i 

Measure  seL  B  - the  twenty-eight  measures  described 

in  the  last  paragraph  of  section  5.3 

Measure  set  C  - the  fifty  measures  described  in  Table  5-2 

Measure  set  D  - the  first  twenty-five  measures  described 

in  Table  5  -2  . 

5.5,1  Preliminary  Configuration  Results 

Tables  5-3  and  5-4  are  tabulations  of  the  training  and  pattern 
classification  results  for  the  parallel  hyperplane  first  stage  configuration. 
For  reasons  which  were  previously  outlined,  it  was  not  expected  that  this 
configuration  should  perform  well  for  patterns  riot  included  in  the  training 
set.  The  tabulation  of  the  first  stage -second  layer  decisions  for  sub- 
segments  (excluding  the  training  sub-set)  indicates  that  this  was  the  case. 
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Table  5-3 

Pattern  Classifier  Results 
Parallel  Ilyperplane  First  Stage  Implementation 


Category  One  Data  --  Measure  Set  A 

Measure  Set  A  (50  measures  category  one  data) 
Number  oi  Tirst  Stage  Hypcrplanes  --109 


First  stage  -  second 

layer 

decisions: 

C/C 

A/C 

C/A 

A/A 

Excluding 

1  6  G 

1 1  4 

101 

109 

Training  sub-set 

Training  sub-set 

1  64 

0 

0 

132 

Total 

330 

114 

101 

241 

Second  stage  dec) si 

ion  s: 

Patlerns 

Machine  No. 

C/C 

A/C 

C/A 

A /A_ 

Considered 

1 

10 

0 

G 

1 

Exc,  training 

1 

Ct 

7 

3 

5 

2 

n 

3 

.9 

1 

3 

4 

It 

4 

7 

3 

3 

4 

I) 

1 

30 

1 

5 

16 

Training  set 

2 

28 

3 

12 

9 

II 

3 

30 

1 

0 

21 

«■ 

4 

28 

3 

5 

1  G 

n 

1 

40 

i 

I  i 

1  7 

Ail 

2 

35 

6 

17 

1  1 

ii 

3 

39 

2 

3 

2  5 

" 

4 

35 

('» 

8 

20 

" 

set 
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l.jblc  5-4 

Pattern  Classifier  Results 
Parallel  Hyperplana  First  Stage  Implementation 

Category  Cne  Data  --  Measure  Set  P 

Number  of  First  Stage  Hyperplatv. s  —  11  1 


First  stage  -  second  'ayer  decisions: 


C/C 

a/g_ 

Q/A. 

A/ A 

Excluding 
Training  sub- 

200 

set 

80 

109 

101 

Troining  sub- 

set  164 

0 

0 

1  32 

Total 

364 

80 

109 

233 

Second  stage 

decisions: 

Patterns 

Machine  No . 

C/C 

A/C 

C/A 

A/A 

Considered 

1 

.1  0 

0 

6 

1 

Excl .  Training 

2 

7 

3 

4 

3 

It 

3 

10 

0 

4 

3 

" 

4 

7 

3 

3 

4 

ii 

1 

31 

0 

5 

1G 

Training  set 

2 

26 

5 

6 

1  5 

•  I 

3 

31 

0 

1 

20 

•  < 

4 

29 

2 

5 

1  G 

it 

1 

41 

u 

i  i 

1  7 

Al  i 

2 

33 

8 

10 

1  8 

" 

3 

41 

U 

5 

2  3 

" 

4 

3  6 

5 

8 

20 

ii 

92 


Tabu.  5-5 


Pattern  Classifier  Results 

Modified  Parallel  llyperplane  Fitst  Stage  Implementation 
Category  One  Data  —  Measure  Sot  A 
Number  of  First  Stage  Ilyperplanes  9 


First  stage  --second  layer  decisions: 


c/c 

A/C 

C/A 

A/A 

Excluding 

Training  sub-set 

20  4 

7  G 

1 1  8 

9  2 

Training  sub-set 

144 

2.0 

4  G 

8  6 

Total 

348 

9G 

1  G  4 

178 

Second  stage  decisions: 

Patterns 

Machine  No . 

C/C 

hi c_ 

C/A 

A/A 

Considered 

1 

8 

2 

5 

2 

Excl .  training 

2 

8 

2 

5 

2 

•  i 

3 

7 

3 

5 

2 

1' 

4 

6 

4 

4 

3 

1 1 

1 

29 

2 

11 

10 

Training  set 

2 

2G 

5 

12 

9 

ii 

3 

28 

3 

9 

1  2 

1 1 

4 

28 

3 

r 

O 

1  5 

" 

i 

37 

A 

-1 

\  r, 

X 

12 

A 1 1 

2 

34 

7 

1  7 

1  1 

■  ■ 

3 

3  5 

G 

1  4 

14 

" 

4 

34 

7 

10 

18 

" 
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Table  5-6 


Pattern  Classifier  Results 

Modified  Parallel  Hyperpl  une  First.  Stage  Implementation 


Category 

One  Data  -- 

Measure 

Set  B 

Number  of 

Tirst  Stage 

Ilyperplanes  --  9 

Tirst  stage  - 

second  layer  decisions: 

Excluding 

C/C 

A/C 

C/A 

A/A 

Training  sub 

-set  217 

63 

130 

80 

Training  sub 

-set  140 

24 

38 

94 

Total 

357 

87 

168 

174 

Second  stage  decisions: 

Patterns 

Machine  No, 

C/C 

A/C 

C/A 

A/A 

Con  sidered 

1 

9 

1 

5 

2 

Excl .  trainin' 

2 

9 

] 

4 

3 

ti 

3 

9 

1 

5 

2 

n 

4 

8 

2 

3 

4 

ii 

1 

27 

4 

6 

1  5 

Training  set 

2 

27 

4 

7 

14 

n 

3 

2/ 

4 

4 

17 

ti 

4 

29 

2 

5 

1  6 

ii 

1 

36 

5 

11 

17 

All 

2 

36 

5 

11 

1  7 

|i 

3 

36 

5 

9 

19 

1* 

4 

37 

4 

8 

2 

II 

94 


Table  5-7 

Pattern  Classitiei  Results 

Fixed  Radius  Ilypcrsphcre  Clustering  Hist  Stage  Implementation 

Category  One  Data  --  Measure  Set  A 

Number  of  first  stage  prototype  poinLs: 

Cough  -  5  Artifact  -  G 


First  stage  -  second  layer  decisions: 


C/C 

A/C 

Oil. L 

A/A 

Excluding 
Training  sub- 

178 

set 

i  02 

G3 

147 

Training  sub- 

set  109 

55 

45 

87 

Tota! 

287 

1  57 

108 

2  34 

Second  stage 

decisions: 

Patterns 

Machine  No. 

c/I! 

A/C 

C/A 

A/A 

Considered 

1 

9 

1 

4 

3 

Excl .  training  set 

2 

8 

2 

5 

2 

3 

9 

1 

4 

3 

" 

4 

9 

1 

2 

5 

" 

1 

2G 

5 

5 

1  G 

Training  set 

2 

2.0 

3 

1  4 

7 

l> 

3 

26 

S 

4 

1  7 

1) 

4 

30 

1 

4 

17 

n 

I 

35 

6 

9 

i  9 

All 

2 

3  6 

5 

19 

9 

li 

3 

35 

G 

8 

20 

•• 

4 

39 

2 

e 

22 

li 

1 

i 
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Table  5-8 


Pattern  Classifier  Results 

Fixed  Radius  Hyperspherc  Clustering  First.  Stage  implementation 


Category 

One  Data 

--  Measure 

_Set_B 

Number  of  first  stage 

;  prototype  points: 

Cough 

-  4 

Artifact  • 

-  a 

Tirst  stage  -  second  layer 

decisions; 

C/C 

A/C 

C/A 

A/A 

Excluding 

190 

90 

4  8 

1  52 

Training  sub- 

set 

Training  sub- 

sc-t  7  5 

89 

40 

92 

Total 

265 

179 

88 

254 

Second  stage 

decisions: 

Patterns 

Machine  No. 

C/C 

A/C 

C/A 

A/A 

Considered 

1 

10 

0 

4 

3 

Fxcl .  Training  set 

2 

10 

0 

2 

5 

n 

3 

10 

0 

4 

3 

ii 

4 

9 

1 

5 

2 

it 

1 

25 

6 

8 

1  3 

Training  set 

2 

2  8 

3 

14 

7 

li 

3 

2  5 

6 

8 

13 

1 1 

4 

28 

3 

3 

18 

li 

1 

35 

6 

12 

1  6 

All 

2 

3  8 

3 

16 

12 

II 

q 

o 

3  5 

6 

12 

16 

II 

4 

3  7 

4 

8 

20 

■  1 
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Table  5-9 

Pattern  Classifier  Results 
Final  Machine  Configuration* 


Category  One  Data  --  Measure  Set  A 


Number  of  first  stage 

prototype 

points: 

Cough 

-  18 

Artifact 

-  13 

First  stage  - 

second  layer 

decision  s: 

C/C 

A/C 

C/A 

A/A 

Fxcluding 

222 

58 

77 

133 

Training  sub 

-set 

Training  sub 

-set  151 

13 

35 

97 

Total 

373 

71 

112 

230 

Second  stage  decisions: 

Patterns 

Machine  No. 

C/C 

A/C 

C/A 

A/A 

Considered 

1 

8 

2 

4 

3 

Excl .  training  set 

2 

9 

1 

5 

2 

" 

3 

7 

3 

4 

3 

li 

4 

8 

2 

2 

5 

il 

5 

6 

4 

3 

4 

1 1 

1 

30 

1 

5 

16 

Training  set 

2 

29 

2 

15 

6 

It 

3 

28 

3 

3 

18 

II 

4 

30 

1 

4 

17 

n 

5 

31 

0 

3 

18 

li 

1 

38 

3 

9 

19 

All 

2 

38 

3 

20 

8 

n 

3 

35 

6 

7 

2] 

1 1 

4 

38 

3 

6 

22 

1 1 

5 

37 

4 

G 

22 

li 

* 

First  stage  prototype  points  computed  by  an  iterative  algorithm. 


S  7 


i  fc 

s  11 


1 

Table  5 

-10 

f 

Pattern  Classi 

fier  Results 

1 

Final 

Machine  Configuration 

1 

Category  One  Data 

--  Measure 

Set  B 

1 

Number  of  first  stage  prototype  points: 

Cough 

16 

Artifact  -  1 

5 

1 

r irst  stage  - 

second  layer  decisions: 

ft 

C/C 

A/C 

C/A 

A/A 

f 

Excluding 

210 

70 

44 

1  66 

l 

Training  sub 

-set 

1 

Training  sub 

-set  137 

27 

1 9 

103 

1 

Total 

347 

97 

73 

2  69 

I 

Second  stage  decisions: 

1 

Patterns 

Machine  No. 

C/C 

A/C 

c/a 

A/A 

Considered 

1 

9 

1 

3 

4 

Excl.  training  set 

2 

9 

1 

5 

2 

" 

3 

9 

1 

0 

7 

1 1 

4 

9 

1 

4 

3 

" 

5 

9 

1 

0 

7 

11 

] 

2  G 

5 

10 

11 

Training  set 

2 

27 

4 

7 

]  4 

1 1 

3 

2G 

c 

u 

1  6 

4 

28 

3 

4 

1  7 

1 i 

b 

29 

2 

5 

1  6 

1 1 

1 

35 

G 

13 

15 

All 

2 

3  6 

5 

]?. 

10 

1 1 

3 

3  5 

G 

5 

23 

If 

4 

37 

4 

0 

20 

I) 

5 

3  8 

3 

5 

23 

" 

Rasp! 


Excl .  training  set 

ft 


M 


training  set 
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Table  5-12 

Pattern  Classifier  Results 
Tinal  Machine  Configuration 

Category  Two  Data  -  Measure  Set  B 

Number  of  first  stage  prototype  points: 
Cough  -  15  Artifact  -  15 


First  stage  -  second  layer  decisions: 


C/C 

A/C 

c/a_ 

A/A 

Excluding 

Training  sub-set 

247 

115 

205 

439 

Training  sub-set 

93 

15 

46 

160 

Total 

340 

1  30 

251 

599 

Second  stage  decisions: 

Patterns 

Machine  No. 

£/c 

A/C 

C/A 

A/A 

Considered 

1 

50 

6 

2 

]  2 

ExcL.  training  set 

2 

44 

12 

3 

1  1 

li 

3 

31 

25 

2 

1  2 

■  i 

4 

40 

10 

1 

1  3 

1 1 

5 

37 

19 

5 

9 

n 

1 

50 

C 

2 

12 

Training  set 

2 

3  G 

20 

5 

9 

n 

3 

37 

19 

2 

1  2 

li 

4 

53 

3 

2 

12 

li 

5 

43 

8 

2 

12 

li 

1 

100 

12 

4 

24 

AH 

2 

80 

32 

8 

20 

li 

3 

08 

44 

4 

24 

" 

4 

99 

1  3 

3 

25 

M 

5 

85 

27 

7 

21 

II 

100 


Table  5-13 


Pattern  Classifier  Results 
Pinal  Machine  Configuration 


Category 

Two  Data  -  Measure  Set  C 

Number  of  first  stage  prototype  points: 

Cough 

-  13  Artifact 

-  14 

I'irst  stage  - 

second  layer  decisions: 

C/C 

A/C  C/A 

Excluding 
Training  sub 

305 

-set 

57  1 80 

458 

Training  sub 

-set  9G 

12  20 

1  86 

Total 

401 

69  206 

644 

Second  stage  decisions: 

Patterns 

Machine  No. 

c/c 

A/C  C/A 

A/A 

Considered 

1 

51 

5  1 

1  3 

Excl ,  training  set 

2 

5G 

C  1 

13 

li 

3 

54 

2  1 

13 

li 

4 

53 

3  1 

13 

ii 

5 

55 

1  2 

12 

1 1 

1 

50 

6  0 

14 

Training  set 

2 

55 

1  1 

1  3 

" 

V' 

40 

10  0 

1  4 

1 1 

4 

55 

1  1 

1  3 

li 

5 

5  0 

0  0 

14 

" 

1 

101 

1 1  1 

2  7 

All 

2 

1 1  1 

1  2 

20 

1 1 

3 

100 

12  1 

2  7 

li 

4 

106 

4  2 

2  6 

1 1 

5 

111 

1  2 

2  6 

II 

Per  cent  coi  i 

ect  classifications  (all  patterns): 

Machine  No 

.  Cough 

Artifact 

Overall 

1 

90.1 

96.4 

91.4 

2 

99.1 

92.8 

97.9 

3 

89.3 

90. 4 

90.7 

4 

90.4 

92.8 

95.7 

5 

99 . 1 

92.8 

9  7.9 
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Table  5-14 

Pattern  Classifier  Results 
Tinal  Machine  Configuration 

Category  Two  Data  -  Measure  Set  D 


Number  of  first  stage 

prototype 

points: 

Cough  - 

15 

Artifact  - 

•  12 

First  stage  -  £ 

second  layer  d 

ecis  ions: 

C/C 

A/C 

C/A 

A/A 

Excluding 
Training  sub- 

303 

set 

59 

1  40 

504 

Training  sub- 

set  9  8 

10 

24 

1  82 

Total 

401 

09 

1  64 

686 

Second  stage 

decisions: 

Patterns 

Machine  No . 

C/C 

A/C 

C/A 

A/A 

Considered 

1 

49 

7 

1 

1  3 

Excl .  training  set 

2 

51 

5 

1 

3  3 

•  i 

3 

50 

6 

1 

13 

M 

4 

51 

5 

1 

1  3 

M 

5 

53 

3 

3 

1 1 

" 

1 

51 

5 

0 

14 

Training  set 

2 

52 

A 

1 

13 

II 

3 

48 

8 

1 

13 

|i 

4 

52 

4 

0 

14 

n 

5 

54 

2 

0 

14 

n 

1 

100 

12 

1 

27 

All 

2 

103 

9 

2 

20 

|i 

3 

98 

14 

2 

2  6 

if 

4 

103 

9 

1 

27 

li 

5 

107 

5 

3 

25 

il 
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All  of  the  preliminary  machines  investigated  show  a  pronounced 
tendency  to  classify  artifacts  as  coughs  when  applied  to  patterns  not  in 
the  training  set. 

The  second  stage  of  the  pattern  classiter  presumes  statistical 
independence  of  the  binary  input.  That  this  assumption  is  flagrantly 
violated  with  respect  to  machines  2  and  4  in  the  parallel  hypeiplane  con¬ 
figuration  is  reflected  in  the  classification  results.  It  is  noted  that, 
although  measure  set  B  is  of  lower  dimensionality  than  measure  saL  A, 
there  is  no  apparent  degradation  of  classifier  performance.  This  can 
probably  Ire  traced  to  approximately  equal  number  of  hyporplanos  imple¬ 
mented  by  the  first  stage  for  both  measure  sets. 

An  overall  evaluation  of  this  configuration  indicates  that  it  is 
somewhat  less  than  satisfactory  for  the  data  with  which  the  research 
described  by  this  dissertation  is  concerned. 

Tables  S-b  and  S-G  are  tabulations  of  the  losults  obtained  for  a 
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resulted  in  a  coll  defined  by  adjacent  hyper  planes  being  deleted  unless 
it  was  occupied  by  at  least  1%  of  the  training  sub-segments. 

It  would  be  expected  that  classification  results  would  be  degraded 
because  of  the  lower  dimensionality  of  the  binary  vector  presented  to 
machines  2  and  4  of  tiro  second  stage  of  tire  pa  Horn  classiticr  and  because 


of  the  increased  number  of  errors  in  the  lir.st  stage-second  layer  decisions, 
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which  would  be  reflected  as  an  increased  error  rats  for  machines  1  and  3. 
That  such  was  the  case  is  apparent  from  a  comparison  of  the  classification 
results  on  the  training  set  for  machine  3. 

Tables  5-7  and  5-8  are  tabulations  of  the  results  achieved  by  the 
preliminary  machine  which  uses  as  a  first  stage  a  minimum  distance 
classifier  witli  respect  tc  point  sets. 

The  prototype  points  are  determined  by  assigning  a  training  sub- 
segment  to  an  existing  cluster  (for  which  the  prototype  points  are  centroids) 
if  it  is  within  a  pre -specified  distance  from  its  centroid  and  then  updating 
the  mean  value  of  the  cluster.  If  no  such  cluster  exists,  the  training  sub- 
segment  is  used  to  define  a  new  cluster  centroid.  This  procedure  was 
reiterated  until  all  training  sub-segments  had  been  assigned  to  a  cluster. 
Cough  and  artifact  training  sub-segments  were  considered  separately.  A 
maximum  of  twenty  clusters  was  allowed  for  each  of  the  classes.  If 
training  sub-segments  existed  foi  winch  clusters  wore  not  found  after  the 
maximum  number  of  allowable  clusters  had  been  formed,  the  radius  of  the 
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fixed  increment  and  the  process  repealed.  This  was  continued  until  all 
training  sub-segmenls  had  been  assigned  to  a  cluster. 

Unless  at  least  seven  of  the  artifact  training  sub-segments  or 
erght  of  the  cough  sub-segments  fell  within  a  cluster  boundary,  the 
centroid  of  the  cluster  was  deleted  from  the  set  of  prototype  points.  The 
results  indicated  that  this  requirement  resulted  in  the  deletion  of  too  many 
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of  the  originally  determined  piototype  points.  Although  improved  results 
could  have  been  expected  had  a  modification  to  the  above  procedure  been 
made,  the  configuration  was  so  similar  to  that  of  the  final  configuration 
selected  that  only  investigation  of  the  latter  was  pursued. 

The  estimated  a  priori  occurrence  of  a  c-  ~.gh  was  0.59  6.  It  was 
noted  that  in  no  case  did  this  a  priori  probability  change  the  decision  which 
would  have  been  made  had  a  maximum  likelihood  criterion  been  used. 

5,5.2  Pinal  Machine  Configuration  Results 

Tables  5-9  through  5-1]  are  taoulations  of  Die  results  of  machine 
training  and  recognition  on  category  one  date.  Tables  5-12  through  5-14 
display  the  results  for  category  two  data.  As  previously  noted,  for  category 
two  data,  the  machine  decisions  are  maximum  likelihood  classifications. 

The  "minimum  distance  with  regard  to  point  sets"  first  stage  configura¬ 
tion  of  this  machine  differed  from  that  of  the  machine  previously  discussed 
only  in  the  manner  in  which  the  prototype  points  were  determined.  As  before, 
a  maximum  of  twenty  each  artifact  and  cough  clusters  were  allowed.  Prior 
to  ti aining , these  forty  clusters  were  assigned  arbitrarily  valued  centroids. 
During  training,  a  sub-segment  was  assigned  to  the  cluster  within  its  class 
(cough  or  artifact)  for  which  the  distance  between  the  sub-segment  and  the 
cluster  centroid  were  closest  (in  the  Euclidian  sense)  compared  to  the  other 
clusters  within  the  class.  The  centroid  of  the  cluster  so  chosen  was  then 


updated  to  include  the  newly  assigned  sub-segment. 
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The  distance  of  the  cluster  centroids  from  the  origin  were  stored 
in  central  memory.  When  a  centroid  was  updated,  the  change  in  its 
distance  from  the  origin  was  computed.  The  training  sub-set  was  pre¬ 
sented  repeatedly  until  the  changes  in  distance  from  the  centroids  of  all 
clusters  to  the  origin  were  negligible,  at  which  time  first  stage  training 
was  complete.  Clusters  which  contained  less  than  three  of  the  training 
sub-segments  were  deleted. 

A  comparison  of  Tables  5-9  and  5-10  again  indicates  that  the 
pattern  set  with  the  lowest  dimensionality  yields  a  higher  percentage  of 
correct  decisions.  This  is  probably  due  to  the  relative  number  of  cough 
and  artifact  prototype  points  implemented  by  the  first  stage  of  the  classi¬ 
fier.  For  the  noisy  category  one  signals,  an  increase  in  dimensionality  of 
the  pattern  results  in  a  greater  percentage  of  pattern  space  being  occupied 
by  cough  patterns  with  a  resultant  increase  in  the  number  of  erroneous 
artifact  classifications. 

in  Table  5-10  it  will  be  noted  that  although  the  classification  for 
patterns  excluding  the  training  set  results  are  quite  acceptable  for 
Machines  3  and  5,  the  overall  training  results  on  the  total  data  set 
indicate  that  if  additional  patterns  were  included  in  the  data  a  highei 
error  rate  would  result. 


The  results  obtained  by  the  use  of  the  measures  contained  in 
measure  sot  C  (which  was  chosen  on  the  basis  of  category  two  data) 
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for  the  classification  of  category  one  data  are  tabulated  in  Table  5-11. 

It  was  not  expected  that  this  set  of  measures  would  yield  as  high  a 
percentage  of  correct  classifications  as  measures  chosen  on  the  basis 
of  category  one  data  if  the  measure  selection  technique  used  was  valid. 

It  will  be  noted  that  such  was  the  case. 

The  same  argument  applies  to  the  case  for  which  the  results  are 
tabulated  in  Table  5-12. 

The  training  and  classification  results  for  category  two  data  with 
respect  to  the  measures  chosen  from  a  consideration  of  category  two  data 
are  listed  in  Tables  5-13  and  5-14.  As  would  normally  be  expected, 
measure  set  C  (fifty  dimensions)  yielded  fewer  erroneous  classifications 
than  did  measure  set  D  (twenty-five  dimensions). 

The  results  indicate  that  second  stage  machines  two,  four  and  five 
appear  to  be  mure  satisfactory  than  machines  one  and  three.  If  the  pattern 
classifier  was  to  be  realized  by  real  time  circuitry,  machine  5  should  be 

.  .  ~  ~  J  f  «  .  ,  „  „  _  ....  J  _ .  .  1  - :  ..  -  -1  f  _  ..  *.  1.  -■  ..  - r :  I  .  mi  • 
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would  result  in  less  complex  circuitry. 


CHAPTER  VI 


CONCLUSIONS 


6.1  Summary  and  Recommendations 

A  model  for  a  pattern  classifier  which  yields  usable  results  has 
been  presented.  The  model  circumvents  the  difiiculties  attached  to  esti¬ 
mating  a  conditional  multi-dimensional  probability  function  by  eifecting 
a  preliminary  piecewise  linear  partition  of  pattern  space  and  assigning 
binary  values  to  the  resulting  cells. 

More  satisfactory  results  were  obtained  for  simulated  data  than 
for  data  recorded  in  the  hospital  environment.  It  was  postulated  that  this 
v/as  due  to  the  presence  of  contaminating  noise  in  the  category  one  data. 
Use  of  the  simulated  data  was  justifiable  since  a  real  time  machine  would 
probably  be  placed  in  close  proximity  to  the  patient  being  monitored.  This 
would  result  in  higher  quality  signals  being  available  at  the  input  to  the 
classifier. 


A  measure  selection  technique  was  examined.  Experimental,  results 
indicate  that  the  procedure  is  adequate  for  a  preliminary  selection  of 
measures,  but  that  a  final  selection  of  a  set  of  measures  should  be  made 
on  an  empirical  basis. 

The  measures  selected  for  use  by  tire  various  classifiers  arc  satis¬ 
factory  for  either  real  time  or  general  purpose  computer  recognition.  In 
the  latter  case,  however,  analog  pic-processing  to  obtain  a  representation 


107 


108 


of  the  spectral  measures  is  appropriate  due  to  the  disproportionate  calculation 
time  required  to  compute  these  measures.  Segmentation  and  thresholding  of 
the  signals  should  be  implemented  prior  to  digit;  ration  of  the  data. 

If  the  present  recording  procedure  is  continued,  an  effort  should  be 
made  to  improve  the  quality  of  the  lesuiting  signal. 

Experimental  results  indicate  that  the  assumption  of  Markoff-1 
dependency  between  the  data  sub-segments  is  a  reasonable  hypothesis. 

The  configuration  based  on  '.his  assumption  (machine  number  5)  may  be 
implemented  more  simply  by  real  time  circuitry  and  requires  fewer  recog¬ 
nition  computations  by  a  general  purpose  machine  than  other  machines  with 
comparable  results.  It  should  therefore  be  chosen  for  the  realization  of 
the  recognition  process.  This  conclusion  would  not  necessarily  be  valid 
if  the  pattern  classifier  was  applied  to  data  originating  from  an  experiment 
other  than  that,  considered  in  this  research. 

6.2  Application  Extensions 

The  model  presented  in  this  paper  is  suitable  for  use  (with 
slight  modification)  with  numerous  types  of  data,  the  primary  requirement 
being  that  sequential  segments  of  data  from  the  same  class  be  available. 

An  application  directly  related  to  that  described  by  this  paper 
would  be  the  classification  of  coughs  as  having  originated  from  a  patient 
suffering  from  irreversible  lung  damage  as  contrasted  with  coughs  emanat¬ 
ing  from  a  respiratory  system  in  which  the  cough  Causing  factors  are  of  a 
temporaiy  or  reversible  nature.  The  physiological  model  indicates  that  a 


difference  in  the  audible  cbaracteii sties  of  the  coughs  (and  possibly  forced 
expirations)  probably  exists . 

A  study  of  the  statistical  characteristics  of  the  measures  calculated 
and  their  relationship  to  the  physiological  model  would  be  rewarding . 
Additionally ,  such  a  study  would  indicate  the  degree  of  corral  alien  between 
the  various  measures.  If  two  or  more  measures  are  highly  correlated  sta¬ 
tistically,  only  one  should  be  used  in  the  recognition  pieces:;. 
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to  Hi  h  pf  ar  probIem  wlth  which  the  research  was  concerned  was  the  development  of  a  technique 
to  discriminate  between  coughs  and  other  audible  phenomena  which  originate  in  a  hospital  environment 

Pattern  recognition  provided  such  a  technique.  Experimental  data  was  available  in  the  form  of  audio  tape 
recordings .  ^ 

Implementation  of  a  Bayes  categorization  decision  requires  knowledge  of  the  underlying  conditional 
joint  probability  density  functions  of  the  measures  which  typify  the  patterns  to  be  recognized.  An  adaptive 
pattern  ciassuier  model  was  presented  which  circumvented  the  difficulty  of  estimating  thes^  functions 
The  model  is  generally  applicable  to  the  two-class  case  in  which  the  patterns  to  be  classified  consist  of 
sequential  segments  of  data  known  to  have  originated  from  the  same  class.  The  model  took  the  form  of  a 
ayered  machine  The  first  stage  was  a  mtnimum  distance  classifier  with  respect  to  point  sets  while  the 
second  stage  utilized  the  first  stage  binary  valued  outputs  to  implement  a  Bayes  decision. 

The  feature  extraction  and  measure  selection  problems  -were  examined  experimentally.  Feature 
calculation  algorithms  were  developed  which  are  generally  applicable  to  time  varying  signals  The 
effectiveness  of  cn  algorithm  for  selection  of  a  set  of  candidate  measures  was  verified. 

-n  ,  Experimental  results  indicated  that,  for  the  particular  data  with  which  the  research  was  concerned, 
an  assumption  of  ^ar^off-1  dependency  between  sequential  first  stage  decisions  of  the  pattern  classifier 

was  warrantee..  „  pattern  ciassuier  which  was  based  on  this  assumption  classified  97.97.  of  th-  pattern^ 
presentee  to  itr  input  correctly.  "  patterns 
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